{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Classification Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll be working with some California Census Data, we'll be trying to use various features of an individual to predict what class of income they belogn in (>50k or <=50k). \n",
    "\n",
    "Here is some information about the data:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table>\n",
    "<thead>\n",
    "<tr>\n",
    "<th>Column Name</th>\n",
    "<th>Type</th>\n",
    "<th>Description</th>\n",
    "</tr>\n",
    "</thead>\n",
    "<tbody>\n",
    "<tr>\n",
    "<td>age</td>\n",
    "<td>Continuous</td>\n",
    "<td>The age of the individual</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>workclass</td>\n",
    "<td>Categorical</td>\n",
    "<td>The type of employer the  individual has (government,  military, private, etc.).</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>fnlwgt</td>\n",
    "<td>Continuous</td>\n",
    "<td>The number of people the census  takers believe that observation  represents (sample weight). This  variable will not be used.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>education</td>\n",
    "<td>Categorical</td>\n",
    "<td>The highest level of education  achieved for that individual.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>education_num</td>\n",
    "<td>Continuous</td>\n",
    "<td>The highest level of education in  numerical form.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>marital_status</td>\n",
    "<td>Categorical</td>\n",
    "<td>Marital status of the individual.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>occupation</td>\n",
    "<td>Categorical</td>\n",
    "<td>The occupation of the individual.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>relationship</td>\n",
    "<td>Categorical</td>\n",
    "<td>Wife, Own-child, Husband,  Not-in-family, Other-relative,  Unmarried.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>race</td>\n",
    "<td>Categorical</td>\n",
    "<td>White, Asian-Pac-Islander,  Amer-Indian-Eskimo, Other, Black.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>gender</td>\n",
    "<td>Categorical</td>\n",
    "<td>Female, Male.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>capital_gain</td>\n",
    "<td>Continuous</td>\n",
    "<td>Capital gains recorded.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>capital_loss</td>\n",
    "<td>Continuous</td>\n",
    "<td>Capital Losses recorded.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>hours_per_week</td>\n",
    "<td>Continuous</td>\n",
    "<td>Hours worked per week.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>native_country</td>\n",
    "<td>Categorical</td>\n",
    "<td>Country of origin of the  individual.</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>income</td>\n",
    "<td>Categorical</td>\n",
    "<td>\"&gt;50K\" or \"&lt;=50K\", meaning  whether the person makes more  than \\$50,000 annually.</td>\n",
    "</tr>\n",
    "</tbody>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Follow the Directions in Bold. If you get stuck, check out the solutions lecture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### THE DATA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Read in the census_data.csv data with pandas**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "census = pd.read_csv(\"census_data.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>education_num</th>\n",
       "      <th>marital_status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>gender</th>\n",
       "      <th>capital_gain</th>\n",
       "      <th>capital_loss</th>\n",
       "      <th>hours_per_week</th>\n",
       "      <th>native_country</th>\n",
       "      <th>income_bracket</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Wife</td>\n",
       "      <td>Black</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>Cuba</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age          workclass   education  education_num       marital_status  \\\n",
       "0   39          State-gov   Bachelors             13        Never-married   \n",
       "1   50   Self-emp-not-inc   Bachelors             13   Married-civ-spouse   \n",
       "2   38            Private     HS-grad              9             Divorced   \n",
       "3   53            Private        11th              7   Married-civ-spouse   \n",
       "4   28            Private   Bachelors             13   Married-civ-spouse   \n",
       "\n",
       "           occupation    relationship    race   gender  capital_gain  \\\n",
       "0        Adm-clerical   Not-in-family   White     Male          2174   \n",
       "1     Exec-managerial         Husband   White     Male             0   \n",
       "2   Handlers-cleaners   Not-in-family   White     Male             0   \n",
       "3   Handlers-cleaners         Husband   Black     Male             0   \n",
       "4      Prof-specialty            Wife   Black   Female             0   \n",
       "\n",
       "   capital_loss  hours_per_week  native_country income_bracket  \n",
       "0             0              40   United-States          <=50K  \n",
       "1             0              13   United-States          <=50K  \n",
       "2             0              40   United-States          <=50K  \n",
       "3             0              40   United-States          <=50K  \n",
       "4             0              40            Cuba          <=50K  "
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "census.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>education_num</th>\n",
       "      <th>capital_gain</th>\n",
       "      <th>capital_loss</th>\n",
       "      <th>hours_per_week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>38.581647</td>\n",
       "      <td>10.080679</td>\n",
       "      <td>1077.648844</td>\n",
       "      <td>87.303830</td>\n",
       "      <td>40.437456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>13.640433</td>\n",
       "      <td>2.572720</td>\n",
       "      <td>7385.292085</td>\n",
       "      <td>402.960219</td>\n",
       "      <td>12.347429</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>17.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>28.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>40.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>37.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>40.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>48.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>45.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>90.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>99999.000000</td>\n",
       "      <td>4356.000000</td>\n",
       "      <td>99.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                age  education_num  capital_gain  capital_loss  hours_per_week\n",
       "count  32561.000000   32561.000000  32561.000000  32561.000000    32561.000000\n",
       "mean      38.581647      10.080679   1077.648844     87.303830       40.437456\n",
       "std       13.640433       2.572720   7385.292085    402.960219       12.347429\n",
       "min       17.000000       1.000000      0.000000      0.000000        1.000000\n",
       "25%       28.000000       9.000000      0.000000      0.000000       40.000000\n",
       "50%       37.000000      10.000000      0.000000      0.000000       40.000000\n",
       "75%       48.000000      12.000000      0.000000      0.000000       45.000000\n",
       "max       90.000000      16.000000  99999.000000   4356.000000       99.000000"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "census.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** TensorFlow won't be able to understand strings as labels, you'll need to use pandas .apply() method to apply a custom function that converts them to 0s and 1s. This might be hard if you aren't very familiar with pandas, so feel free to take a peek at the solutions for this part.**\n",
    "\n",
    "** Convert the Label column to 0s and 1s instead of strings.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([' <=50K', ' >50K'], dtype=object)"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "census['income_bracket'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "def label_fix(label):\n",
    "    if label==' <=50K':\n",
    "        return 0\n",
    "    else:\n",
    "        return 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "census['income_bracket'] = census['income_bracket'].apply(label_fix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>education_num</th>\n",
       "      <th>capital_gain</th>\n",
       "      <th>capital_loss</th>\n",
       "      <th>hours_per_week</th>\n",
       "      <th>income_bracket</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>38.581647</td>\n",
       "      <td>10.080679</td>\n",
       "      <td>1077.648844</td>\n",
       "      <td>87.303830</td>\n",
       "      <td>40.437456</td>\n",
       "      <td>0.240810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>13.640433</td>\n",
       "      <td>2.572720</td>\n",
       "      <td>7385.292085</td>\n",
       "      <td>402.960219</td>\n",
       "      <td>12.347429</td>\n",
       "      <td>0.427581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>17.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>28.000000</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>37.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>48.000000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>90.000000</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>99999.000000</td>\n",
       "      <td>4356.000000</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                age  education_num  capital_gain  capital_loss  \\\n",
       "count  32561.000000   32561.000000  32561.000000  32561.000000   \n",
       "mean      38.581647      10.080679   1077.648844     87.303830   \n",
       "std       13.640433       2.572720   7385.292085    402.960219   \n",
       "min       17.000000       1.000000      0.000000      0.000000   \n",
       "25%       28.000000       9.000000      0.000000      0.000000   \n",
       "50%       37.000000      10.000000      0.000000      0.000000   \n",
       "75%       48.000000      12.000000      0.000000      0.000000   \n",
       "max       90.000000      16.000000  99999.000000   4356.000000   \n",
       "\n",
       "       hours_per_week  income_bracket  \n",
       "count    32561.000000    32561.000000  \n",
       "mean        40.437456        0.240810  \n",
       "std         12.347429        0.427581  \n",
       "min          1.000000        0.000000  \n",
       "25%         40.000000        0.000000  \n",
       "50%         40.000000        0.000000  \n",
       "75%         45.000000        0.000000  \n",
       "max         99.000000        1.000000  "
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "census.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [],
   "source": [
    "#lambda label :int(label=='<=50K')\n",
    "#census['income_bracket'].apply(lambda label: int(label==' <=50K'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>education_num</th>\n",
       "      <th>marital_status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>gender</th>\n",
       "      <th>capital_gain</th>\n",
       "      <th>capital_loss</th>\n",
       "      <th>hours_per_week</th>\n",
       "      <th>native_country</th>\n",
       "      <th>income_bracket</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>United-States</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Wife</td>\n",
       "      <td>Black</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>Cuba</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age          workclass   education  education_num       marital_status  \\\n",
       "0   39          State-gov   Bachelors             13        Never-married   \n",
       "1   50   Self-emp-not-inc   Bachelors             13   Married-civ-spouse   \n",
       "2   38            Private     HS-grad              9             Divorced   \n",
       "3   53            Private        11th              7   Married-civ-spouse   \n",
       "4   28            Private   Bachelors             13   Married-civ-spouse   \n",
       "\n",
       "           occupation    relationship    race   gender  capital_gain  \\\n",
       "0        Adm-clerical   Not-in-family   White     Male          2174   \n",
       "1     Exec-managerial         Husband   White     Male             0   \n",
       "2   Handlers-cleaners   Not-in-family   White     Male             0   \n",
       "3   Handlers-cleaners         Husband   Black     Male             0   \n",
       "4      Prof-specialty            Wife   Black   Female             0   \n",
       "\n",
       "   capital_loss  hours_per_week  native_country  income_bracket  \n",
       "0             0              40   United-States               0  \n",
       "1             0              13   United-States               0  \n",
       "2             0              40   United-States               0  \n",
       "3             0              40   United-States               0  \n",
       "4             0              40            Cuba               0  "
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "census.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perform a Train Test Split on the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['age', 'workclass', 'education', 'education_num', 'marital_status',\n",
       "       'occupation', 'relationship', 'race', 'gender', 'capital_gain',\n",
       "       'capital_loss', 'hours_per_week', 'native_country', 'income_bracket'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "census.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = census.drop(labels= 'income_bracket', axis=1)\n",
    "y = census['income_bracket']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>education_num</th>\n",
       "      <th>marital_status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>gender</th>\n",
       "      <th>capital_gain</th>\n",
       "      <th>capital_loss</th>\n",
       "      <th>hours_per_week</th>\n",
       "      <th>native_country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Wife</td>\n",
       "      <td>Black</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>Cuba</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age          workclass   education  education_num       marital_status  \\\n",
       "0   39          State-gov   Bachelors             13        Never-married   \n",
       "1   50   Self-emp-not-inc   Bachelors             13   Married-civ-spouse   \n",
       "2   38            Private     HS-grad              9             Divorced   \n",
       "3   53            Private        11th              7   Married-civ-spouse   \n",
       "4   28            Private   Bachelors             13   Married-civ-spouse   \n",
       "\n",
       "           occupation    relationship    race   gender  capital_gain  \\\n",
       "0        Adm-clerical   Not-in-family   White     Male          2174   \n",
       "1     Exec-managerial         Husband   White     Male             0   \n",
       "2   Handlers-cleaners   Not-in-family   White     Male             0   \n",
       "3   Handlers-cleaners         Husband   Black     Male             0   \n",
       "4      Prof-specialty            Wife   Black   Female             0   \n",
       "\n",
       "   capital_loss  hours_per_week  native_country  \n",
       "0             0              40   United-States  \n",
       "1             0              13   United-States  \n",
       "2             0              40   United-States  \n",
       "3             0              40   United-States  \n",
       "4             0              40            Cuba  "
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Encode Label to string features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lableEncode(features):\n",
    "    le = LabelEncoder()\n",
    "    return le.fit_transform(features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {},
   "outputs": [],
   "source": [
    "X['workclass'] = lableEncode(X['workclass'])\n",
    "X['education'] = lableEncode(X['education'])\n",
    "X['marital_status'] = lableEncode(X['marital_status'])\n",
    "X['occupation'] = lableEncode(X['occupation'])\n",
    "X['relationship'] = lableEncode(X['relationship'])\n",
    "X['race'] = lableEncode(X['race'])\n",
    "X['gender'] = lableEncode(X['gender'])\n",
    "X['native_country'] = lableEncode(X['native_country'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a4140c358>"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD8CAYAAAB+UHOxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFfVJREFUeJzt3W2MXOV5xvH/FRzAsVO/AFm5tltTYYEILgavjFHSaI0bYyCKUUUQkRUW5NZfnBQqV41plbrhRXWkEApSg2TFbkya4FAnFMuhkK3xqE0lDDgYjDHUGzDBK2Mn2DhdIKRL736YZ8lkY3tmdmbPzPBcP2m05zznOefc87bXnJc5o4jAzMzy84FWF2BmZq3hADAzy5QDwMwsUw4AM7NMOQDMzDLlADAzy5QDwMwsUw4AM7NMOQDMzDI1rtUFnMyZZ54Zs2bNKmx9b775JhMmTChsfaPVCXV2Qo3gOputE+rshBqhsTp37tz584g4q2rHiGjb27x586JI27dvL3R9o9UJdXZCjRGus9k6oc5OqDGisTqBp6KG/7HeBWRmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlqm2vhSEdY5Zq3/wG+Or5gxxw4i2sbB/7VVjvg6z9ytvAZiZZaqmAJA0WdJmSS9I2ivpUklTJfVJ2pf+Tkl9JekeSf2SnpV0ccVyelP/fZJ6x+pOmZlZdbVuAdwNPBIR5wEXAnuB1cC2iJgNbEvjAFcAs9NtBXAvgKSpwBrgEmA+sGY4NMzMrHhVjwFImgR8ArgBICJ+BfxK0lKgJ3XbCJSALwJLgfvSFekeT1sP01Lfvog4kpbbBywB7m/e3cnbyP3wZmYno/L/6ZN0kOYC64DnKX/63wncBAxExOTUR8DRiJgsaSuwNiJ+lKZtoxwMPcDpEXF7av8S8HZEfHXE+lZQ3nKgq6tr3qZNm5p0V6sbHBxk4sSJha1vtE5U5+6BYy2o5vi6xsOht8d+PXOmT2po/k5/zttNJ9TZCTVCY3UuXLhwZ0R0V+tXy1lA44CLgS9ExA5Jd/Pr3T0ARERIOnmS1Cgi1lEOHLq7u6Onp6cZi61JqVSiyPWN1onqLOKsm1qtmjPEnbvH/iSz/ct6Gpq/05/zdtMJdXZCjVBMnbUcAzgAHIiIHWl8M+VAOJR27ZD+Hk7TB4CZFfPPSG0najczsxaoGgAR8RrwqqRzU9MiyruDtgDDZ/L0Ag+l4S3A9elsoAXAsYg4CDwKLJY0JR38XZzazMysBWrdRv8C8G1JpwIvATdSDo8HJC0HXgGuTX0fBq4E+oG3Ul8i4oik24AnU79bhw8Iv9+M9cHYor5kZWbvbzUFQETsAo53QGHRcfoGsPIEy9kAbKinQDMzGxv+JrCZWaYcAGZmmXIAmJllygFgZpYpB4CZWaYcAGZmmXIAmJllygFgZpYpB4CZWabe178JXO8lGXyJBTPLibcAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8tUTQEgab+k3ZJ2SXoqtU2V1CdpX/o7JbVL0j2S+iU9K+niiuX0pv77JPWOzV0yM7Na1LMFsDAi5kZEdxpfDWyLiNnAtjQOcAUwO91WAPdCOTCANcAlwHxgzXBomJlZ8RrZBbQU2JiGNwJXV7TfF2WPA5MlTQMuB/oi4khEHAX6gCUNrN/MzBpQawAE8ENJOyWtSG1dEXEwDb8GdKXh6cCrFfMeSG0najczsxao9SchPx4RA5I+AvRJeqFyYkSEpGhGQSlgVgB0dXVRKpVGvaxVc4bq6t81vv55WqET6iyqxkZeHwCDg4MNL6MIrrN5OqFGKKbOmgIgIgbS38OSHqS8D/+QpGkRcTDt4jmcug8AMytmn5HaBoCeEe2l46xrHbAOoLu7O3p6ekZ2qVm9v++7as4Qd+5u/59J7oQ6i6px/7KehuYvlUo08horiutsnk6oEYqps+ouIEkTJH14eBhYDDwHbAGGz+TpBR5Kw1uA69PZQAuAY2lX0aPAYklT0sHfxanNzMxaoJaPaF3Ag5KG+38nIh6R9CTwgKTlwCvAtan/w8CVQD/wFnAjQEQckXQb8GTqd2tEHGnaPTEzs7pUDYCIeAm48DjtrwOLjtMewMoTLGsDsKH+Ms3MrNn8TWAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0zVHACSTpH0tKStafxsSTsk9Uv6rqRTU/tpabw/TZ9VsYxbUvuLki5v9p0xM7Pa1bMFcBOwt2L8K8BdEXEOcBRYntqXA0dT+12pH5LOB64DPgosAb4u6ZTGyjczs9GqKQAkzQCuAr6RxgVcBmxOXTYCV6fhpWmcNH1R6r8U2BQR70TEy0A/ML8Zd8LMzOqniKjeSdoM/D3wYeAvgRuAx9OnfCTNBP4tIi6Q9BywJCIOpGk/AS4B/i7N88+pfX2aZ/OIda0AVgB0dXXN27Rp06jv3O6BY3X17xoPh94e9eoK0wl1FlXjnOmTGpp/cHCQiRMnNqmaseM6m6cTaoTG6ly4cOHOiOiu1m9ctQ6SPgUcjoidknpGVU0dImIdsA6gu7s7enpGv8obVv+grv6r5gxx5+6qD0nLdUKdRdW4f1lPQ/OXSiUaeY0VxXU2TyfUCMXUWcs79GPApyVdCZwO/A5wNzBZ0riIGAJmAAOp/wAwEzggaRwwCXi9on1Y5TxmozKrzpAfadWcobo/KAzbv/aqhtZt1mpVjwFExC0RMSMiZlE+iPtYRCwDtgPXpG69wENpeEsaJ01/LMr7mbYA16WzhM4GZgNPNO2emJlZXRrZRv8isEnS7cDTwPrUvh74lqR+4Ajl0CAi9kh6AHgeGAJWRsS7DazfzMwaUFcAREQJKKXhlzjOWTwR8UvgMyeY/w7gjnqLNDOz5vM3gc3MMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0xVDQBJp0t6QtIzkvZI+nJqP1vSDkn9kr4r6dTUfloa70/TZ1Us65bU/qKky8fqTpmZWXW1bAG8A1wWERcCc4ElkhYAXwHuiohzgKPA8tR/OXA0td+V+iHpfOA64KPAEuDrkk5p5p0xM7PaVQ2AKBtMox9MtwAuAzan9o3A1Wl4aRonTV8kSal9U0S8ExEvA/3A/KbcCzMzq1tNxwAknSJpF3AY6AN+ArwREUOpywFgehqeDrwKkKYfA86obD/OPGZmVrBxtXSKiHeBuZImAw8C541VQZJWACsAurq6KJVKo17WqjlD1TtV6Bpf/zyt0Al1dkKN0Fidjbw26zU4OFjo+karE+rshBqhmDprCoBhEfGGpO3ApcBkSePSp/wZwEDqNgDMBA5IGgdMAl6vaB9WOU/lOtYB6wC6u7ujp6enrjtU6YbVP6ir/6o5Q9y5u66HpCU6oc5OqBEaq3P/sp7mFnMSpVKJRt4LRemEOjuhRiimzlrOAjorffJH0njgk8BeYDtwTerWCzyUhrekcdL0xyIiUvt16Syhs4HZwBPNuiNmZlafWj76TAM2pjN2PgA8EBFbJT0PbJJ0O/A0sD71Xw98S1I/cITymT9ExB5JDwDPA0PAyrRryczMWqBqAETEs8BFx2l/ieOcxRMRvwQ+c4Jl3QHcUX+ZZmbWbP4msJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZptr/al1mbWpWnRcbbMSqOUPvXdxw/9qrCluvvb95C8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwyVTUAJM2UtF3S85L2SLoptU+V1CdpX/o7JbVL0j2S+iU9K+niimX1pv77JPWO3d0yM7NqatkCGAJWRcT5wAJgpaTzgdXAtoiYDWxL4wBXALPTbQVwL5QDA1gDXALMB9YMh4aZmRWvagBExMGI+HEa/h9gLzAdWApsTN02Alen4aXAfVH2ODBZ0jTgcqAvIo5ExFGgD1jS1HtjZmY1q+sYgKRZwEXADqArIg6mSa8BXWl4OvBqxWwHUtuJ2s3MrAVq/k1gSROB7wE3R8QvJL03LSJCUjSjIEkrKO86oquri1KpNOplrZozVFf/rvH1z9MKnVBnJ9QInVlnI++JsTY4ONjW9UFn1AjF1FlTAEj6IOV//t+OiO+n5kOSpkXEwbSL53BqHwBmVsw+I7UNAD0j2ksj1xUR64B1AN3d3dHT0zOyS81uqPNHu1fNGeLO3TVnYst0Qp2dUCN0Zp37l/W0tpiTKJVKNPKeLUIn1AjF1FnLWUAC1gN7I+JrFZO2AMNn8vQCD1W0X5/OBloAHEu7ih4FFkuakg7+Lk5tZmbWArV89PkY8Dlgt6Rdqe2vgbXAA5KWA68A16ZpDwNXAv3AW8CNABFxRNJtwJOp360RcaQp98LMzOpWNQAi4keATjB50XH6B7DyBMvaAGyop0AzMxsb/iawmVmmHABmZplyAJiZZcoBYGaWKQeAmVmmHABmZplyAJiZZcoBYGaWqfa/CIqZ/YZZdV7jqpn2r72qZeu25vMWgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlikHgJlZphwAZmaZcgCYmWXKAWBmlqmqASBpg6TDkp6raJsqqU/SvvR3SmqXpHsk9Ut6VtLFFfP0pv77JPWOzd0xM7Na1bIF8E1gyYi21cC2iJgNbEvjAFcAs9NtBXAvlAMDWANcAswH1gyHhpmZtUbVAIiI/wCOjGheCmxMwxuBqyva74uyx4HJkqYBlwN9EXEkIo4Cffx2qJiZWYEUEdU7SbOArRFxQRp/IyImp2EBRyNisqStwNqI+FGatg34ItADnB4Rt6f2LwFvR8RXj7OuFZS3Hujq6pq3adOmUd+53QPH6urfNR4OvT3q1RWmE+rshBrBddZrzvRJJ50+ODjIxIkTC6pmdDqhRmiszoULF+6MiO5q/Rr+UfiICEnVU6T25a0D1gF0d3dHT0/PqJd1Q50/nr1qzhB37m74IRlznVBnJ9QIrrNe+5f1nHR6qVSikfdsETqhRiimztGeBXQo7doh/T2c2geAmRX9ZqS2E7WbmVmLjDYAtgDDZ/L0Ag9VtF+fzgZaAByLiIPAo8BiSVPSwd/Fqc3MzFqk6jalpPsp78M/U9IBymfzrAUekLQceAW4NnV/GLgS6AfeAm4EiIgjkm4Dnkz9bo2IkQeWzcysQFUDICI+e4JJi47TN4CVJ1jOBmBDXdWZmdmY8TeBzcwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFMOADOzTDkAzMwy5QAwM8uUA8DMLFOtv8C4mXWMWVV+Y2PVnKG6f4ejFvvXXtX0ZZq3AMzMsuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlAPAzCxTDgAzs0w5AMzMMuUAMDPLlC8FYWZtr9olKOpRz+Uq3u+XoPAWgJlZpgoPAElLJL0oqV/S6qLXb2ZmZYUGgKRTgH8ErgDOBz4r6fwiazAzs7KijwHMB/oj4iUASZuApcDzBddhZlZVM4891OubSyaM+TqK3gU0HXi1YvxAajMzs4IpIopbmXQNsCQi/jSNfw64JCI+X9FnBbAijZ4LvFhYgXAm8PMC1zdanVBnJ9QIrrPZOqHOTqgRGqvz9yPirGqdit4FNADMrBifkdreExHrgHVFFjVM0lMR0d2KddejE+rshBrBdTZbJ9TZCTVCMXUWvQvoSWC2pLMlnQpcB2wpuAYzM6PgLYCIGJL0eeBR4BRgQ0TsKbIGMzMrK/ybwBHxMPBw0eutUUt2PY1CJ9TZCTWC62y2TqizE2qEAuos9CCwmZm1D18KwswsU1kGgKSZkrZLel7SHkk3pfapkvok7Ut/p7S4ztMlPSHpmVTnl1P72ZJ2pMtpfDcdUG85SadIelrS1jTednVK2i9pt6Rdkp5Kbe32vE+WtFnSC5L2Srq0DWs8Nz2Gw7dfSLq53epMtf5Fev88J+n+9L5qq9empJtSfXsk3ZzaxvyxzDIAgCFgVUScDywAVqZLUqwGtkXEbGBbGm+ld4DLIuJCYC6wRNIC4CvAXRFxDnAUWN7CGivdBOytGG/XOhdGxNyKU+za7Xm/G3gkIs4DLqT8mLZVjRHxYnoM5wLzgLeAB2mzOiVNB/4c6I6ICyiffHIdbfTalHQB8GeUr5RwIfApSedQxGMZEdnfgIeAT1L+0tm01DYNeLHVtVXU+CHgx8AllL8cMi61Xwo82gb1zUgv0suArYDatM79wJkj2trmeQcmAS+Tjs+1Y43HqXkx8F/tWCe/vvrAVMonvWwFLm+n1ybwGWB9xfiXgL8q4rHMdQvgPZJmARcBO4CuiDiYJr0GdLWorPek3Sq7gMNAH/AT4I2IGEpd2uVyGv9A+UX7f2n8DNqzzgB+KGln+tY5tNfzfjbwM+Cf0u60b0iaQHvVONJ1wP1puK3qjIgB4KvAT4GDwDFgJ+312nwO+CNJZ0j6EHAl5S/MjvljmXUASJoIfA+4OSJ+UTktyrHb8lOkIuLdKG9mz6C8iXhei0v6LZI+BRyOiJ2trqUGH4+IiylfkXalpE9UTmyD530ccDFwb0RcBLzJiE3/NqjxPWnf+aeBfxk5rR3qTPvNl1IO1t8FJgBLWlnTSBGxl/IuqR8CjwC7gHdH9BmTxzLbAJD0Qcr//L8dEd9PzYckTUvTp1H+1N0WIuINYDvlzdXJkoa/w/Fbl9NogY8Bn5a0H9hEeTfQ3bRfncOfCImIw5T3Wc+nvZ73A8CBiNiRxjdTDoR2qrHSFcCPI+JQGm+3Ov8YeDkifhYR/wt8n/Lrta1emxGxPiLmRcQnKB+T+G8KeCyzDABJAtYDeyPiaxWTtgC9abiX8rGBlpF0lqTJaXg85eMUeykHwTWpW8vrjIhbImJGRMyivDvgsYhYRpvVKWmCpA8PD1Ped/0cbfS8R8RrwKuSzk1NiyhfLr1tahzhs/x69w+0X50/BRZI+lB63w8/nu322vxI+vt7wJ8A36GIx7JVBz5aeQM+Tnlz6lnKm1u7KO93O4Pygcx9wL8DU1tc5x8CT6c6nwP+NrX/AfAE0E950/u0Vj+mFTX3AFvbsc5UzzPptgf4m9Tebs/7XOCp9Lz/KzCl3WpMdU4AXgcmVbS1Y51fBl5I76FvAae14WvzPykH0zPAoqIeS38T2MwsU1nuAjIzMweAmVm2HABmZplyAJiZZcoBYGaWKQeAmVmmHABmZplyAJiZZer/AdoMZU8nuLXwAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X[\"age\"].hist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [],
   "source": [
    "#bins = [0,30,50,70,100]\n",
    "#labels =[0,1,2,3]\n",
    "#X[\"age_buckets\"] = pd.cut(X[\"age\"],bins=bins, labels=labels, include_lowest=True)\n",
    "#X = X.drop(\"age\", axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>education_num</th>\n",
       "      <th>marital_status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>gender</th>\n",
       "      <th>capital_gain</th>\n",
       "      <th>capital_loss</th>\n",
       "      <th>hours_per_week</th>\n",
       "      <th>native_country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>7</td>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>6</td>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>4</td>\n",
       "      <td>11</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>7</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "      <td>2</td>\n",
       "      <td>10</td>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  workclass  education  education_num  marital_status  occupation  \\\n",
       "0   39          7          9             13               4           1   \n",
       "1   50          6          9             13               2           4   \n",
       "2   38          4         11              9               0           6   \n",
       "3   53          4          1              7               2           6   \n",
       "4   28          4          9             13               2          10   \n",
       "\n",
       "   relationship  race  gender  capital_gain  capital_loss  hours_per_week  \\\n",
       "0             1     4       1          2174             0              40   \n",
       "1             0     4       1             0             0              13   \n",
       "2             1     4       1             0             0              40   \n",
       "3             0     2       1             0             0              40   \n",
       "4             5     2       0             0             0              40   \n",
       "\n",
       "   native_country  \n",
       "0              39  \n",
       "1              39  \n",
       "2              39  \n",
       "3              39  \n",
       "4               5  "
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1a41727160>"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAD8CAYAAACcjGjIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAEpBJREFUeJzt3X+s3fV93/Hnqzh0lCSFxOsVA29mqivNDRqhV8RVpu2mTGCYFFM1imBpcVIUVw1U7Yamuu0fRKGRgiZSiSilcxQLM5EQmraz1Tj1LMYV6lRTnIZiIMu4JU6x68AaE1IHNamz9/44X7MTf665x+cen3N/PB/S0f2e9/fz/X4/72vj1/3+uIdUFZIk9fuhSU9AkrT0GA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqrJn0BIa1du3aWr9+/VDbfuc73+HCCy8c7YSWOHteHVZbz6utX1h8z1/60pf+tqr+8ULjlm04rF+/noMHDw617ezsLDMzM6Od0BJnz6vDaut5tfULi+85ydcHGedlJUlSw3CQJDUMB0lSY8FwSLIuyaNJnk3yTJJf7eofTnI0yZPd64a+bX4jyVySrya5rq++uavNJdneV788yeNd/XNJzh91o5KkwQ1y5nASuKOqNgKbgNuSbOzW/U5VXdm99gJ0624CfhLYDPxukvOSnAd8Erge2Ajc3Lefu7t9/TjwMnDriPqTJA1hwXCoqmNV9Rfd8t8BXwEufZ1NtgAPVdV3q+prwBxwdfeaq6rnq+p7wEPAliQBfgb4fLf9LuDGYRuSJC3eWT3KmmQ98HbgceCdwO1JbgEO0ju7eJlecBzo2+wI/z9MXjit/g7grcC3qurkPONPP/42YBvA1NQUs7OzZzP915w4cWLobZcre14dVlvPq61fGF/PA4dDkjcCfwD8WlV9O8l9wF1AdV/vAX7xnMyyU1U7gB0A09PTNeyzvj4bvTrY88q32vqF8fU8UDgkeQO9YHiwqv4QoKpe7Fv/KeCPu7dHgXV9m1/W1ThD/ZvARUnWdGcP/eMlSROwYDh09wQ+DXylqj7eV7+kqo51b38WeLpb3gN8JsnHgX8CbAD+HAiwIcnl9P7xvwn491VVSR4F3kPvPsRWYPcomjuTQ0df4f3bv3AuDzGvwx/7d2M/piQNY5Azh3cCvwAcSvJkV/tNek8bXUnvstJh4JcAquqZJA8Dz9J70um2qvo+QJLbgX3AecDOqnqm29+vAw8l+W3gy/TCSJI0IQuGQ1X9Kb2f+k+393W2+Sjw0Xnqe+fbrqqep/c0kyRpCfA3pCVJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJjQXDIcm6JI8meTbJM0l+tau/Jcn+JM91Xy/u6klyb5K5JE8luapvX1u78c8l2dpX/6kkh7pt7k2Sc9GsJGkwg5w5nATuqKqNwCbgtiQbge3AI1W1AXikew9wPbChe20D7oNemAB3Au8ArgbuPBUo3ZgP9m23efGtSZKGtWA4VNWxqvqLbvnvgK8AlwJbgF3dsF3Ajd3yFuCB6jkAXJTkEuA6YH9VHa+ql4H9wOZu3Zur6kBVFfBA374kSROw5mwGJ1kPvB14HJiqqmPdqm8AU93ypcALfZsd6WqvVz8yT32+42+jdzbC1NQUs7OzZzP910xdAHdccXKobRdj2PmOwokTJyZ6/Emw55VvtfUL4+t54HBI8kbgD4Bfq6pv998WqKpKUudgfj+gqnYAOwCmp6drZmZmqP184sHd3HPorHJxJA6/b2bsxzxldnaWYb9fy5U9r3yrrV8YX88DPa2U5A30guHBqvrDrvxid0mI7utLXf0osK5v88u62uvVL5unLkmakEGeVgrwaeArVfXxvlV7gFNPHG0FdvfVb+meWtoEvNJdftoHXJvk4u5G9LXAvm7dt5Ns6o51S9++JEkTMMi1lXcCvwAcSvJkV/tN4GPAw0luBb4OvLdbtxe4AZgDXgU+AFBVx5PcBTzRjftIVR3vlj8E3A9cAHyxe0mSJmTBcKiqPwXO9HsH18wzvoDbzrCvncDOeeoHgbctNBdJ0nj4G9KSpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqGA6SpIbhIElqLBgOSXYmeSnJ0321Dyc5muTJ7nVD37rfSDKX5KtJruurb+5qc0m299UvT/J4V/9ckvNH2aAk6ewNcuZwP7B5nvrvVNWV3WsvQJKNwE3AT3bb/G6S85KcB3wSuB7YCNzcjQW4u9vXjwMvA7cupiFJ0uItGA5V9RhwfMD9bQEeqqrvVtXXgDng6u41V1XPV9X3gIeALUkC/Azw+W77XcCNZ9mDJGnEFnPP4fYkT3WXnS7uapcCL/SNOdLVzlR/K/Ctqjp5Wl2SNEFrhtzuPuAuoLqv9wC/OKpJnUmSbcA2gKmpKWZnZ4faz9QFcMcVJxceOGLDzncUTpw4MdHjT4I9r3yrrV8YX89DhUNVvXhqOcmngD/u3h4F1vUNvayrcYb6N4GLkqzpzh76x8933B3ADoDp6emamZkZZvp84sHd3HNo2Fwc3uH3zYz9mKfMzs4y7PdrubLnlW+19Qvj63moy0pJLul7+7PAqSeZ9gA3JfnhJJcDG4A/B54ANnRPJp1P76b1nqoq4FHgPd32W4Hdw8xJkjQ6C/74nOSzwAywNskR4E5gJsmV9C4rHQZ+CaCqnknyMPAscBK4raq+3+3ndmAfcB6ws6qe6Q7x68BDSX4b+DLw6ZF1J0kayoLhUFU3z1M+4z/gVfVR4KPz1PcCe+epP0/vaSZJ0hLhb0hLkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhoLhkOSnUleSvJ0X+0tSfYnea77enFXT5J7k8wleSrJVX3bbO3GP5dka1/9p5Ic6ra5N0lG3aQk6ewMcuZwP7D5tNp24JGq2gA80r0HuB7Y0L22AfdBL0yAO4F3AFcDd54KlG7MB/u2O/1YkqQxWzAcquox4Php5S3Arm55F3BjX/2B6jkAXJTkEuA6YH9VHa+ql4H9wOZu3Zur6kBVFfBA374kSRMy7D2Hqao61i1/A5jqli8FXugbd6SrvV79yDx1SdIErVnsDqqqktQoJrOQJNvoXa5iamqK2dnZofYzdQHcccXJEc5sMMPOdxROnDgx0eNPgj2vfKutXxhfz8OGw4tJLqmqY92loZe6+lFgXd+4y7raUWDmtPpsV79snvHzqqodwA6A6enpmpmZOdPQ1/WJB3dzz6FF5+JZO/y+mbEf85TZ2VmG/X4tV/a88q22fmF8PQ97WWkPcOqJo63A7r76Ld1TS5uAV7rLT/uAa5Nc3N2IvhbY1637dpJN3VNKt/TtS5I0IQv++Jzks/R+6l+b5Ai9p44+Bjyc5Fbg68B7u+F7gRuAOeBV4AMAVXU8yV3AE924j1TVqZvcH6L3RNQFwBe7lyRpghYMh6q6+QyrrplnbAG3nWE/O4Gd89QPAm9baB6SpPHxN6QlSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSY3x/x9vJGkFWL/9CxM57v2bLxzLcTxzkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1FhUOSQ4nOZTkySQHu9pbkuxP8lz39eKuniT3JplL8lSSq/r2s7Ub/1ySrYtrSZK0WKM4c3hXVV1ZVdPd++3AI1W1AXikew9wPbChe20D7oNemAB3Au8ArgbuPBUokqTJOBeXlbYAu7rlXcCNffUHqucAcFGSS4DrgP1VdbyqXgb2A5vPwbwkSQNabDgU8N+TfCnJtq42VVXHuuVvAFPd8qXAC33bHulqZ6pLkiZkzSK3/1dVdTTJjwH7k/yv/pVVVUlqkcd4TRdA2wCmpqaYnZ0daj9TF8AdV5wc1bQGNux8R+HEiRMTPf4k2PPKN8l+J/FvCIyv50WFQ1Ud7b6+lOSP6N0zeDHJJVV1rLts9FI3/Ciwrm/zy7raUWDmtPrsGY63A9gBMD09XTMzM/MNW9AnHtzNPYcWm4tn7/D7ZsZ+zFNmZ2cZ9vu1XNnzyjfJft+//QsTOe79my8cS89DX1ZKcmGSN51aBq4Fngb2AKeeONoK7O6W9wC3dE8tbQJe6S4/7QOuTXJxdyP62q4mSZqQxfz4PAX8UZJT+/lMVf1JkieAh5PcCnwdeG83fi9wAzAHvAp8AKCqjie5C3iiG/eRqjq+iHlJkhZp6HCoqueBfzlP/ZvANfPUC7jtDPvaCewcdi6SpNHyN6QlSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSQ3DQZLUMBwkSY0lEw5JNif5apK5JNsnPR9JWs2WRDgkOQ/4JHA9sBG4OcnGyc5KklavJREOwNXAXFU9X1XfAx4Ctkx4TpK0ai2VcLgUeKHv/ZGuJkmagDWTnsDZSLIN2Na9PZHkq0Puai3wt6OZ1eBy97iP+AMm0vOE2fPKt9r65V13L7rnfzbIoKUSDkeBdX3vL+tqP6CqdgA7FnuwJAeranqx+1lO7Hl1WG09r7Z+YXw9L5XLSk8AG5JcnuR84CZgz4TnJEmr1pI4c6iqk0luB/YB5wE7q+qZCU9LklatJREOAFW1F9g7psMt+tLUMmTPq8Nq63m19Qtj6jlVNY7jSJKWkaVyz0GStISs6HBY6CM5kvxwks916x9Psn78sxydAfr9j0meTfJUkkeSDPRI21I26MeuJPm5JJVk2T/ZMkjPSd7b/Vk/k+Qz457jqA3wd/ufJnk0yZe7v983TGKeo5JkZ5KXkjx9hvVJcm/3/XgqyVUjn0RVrcgXvRvbfwX8c+B84C+BjaeN+RDwe93yTcDnJj3vc9zvu4Af6ZZ/eTn3O2jP3bg3AY8BB4DpSc97DH/OG4AvAxd3739s0vMeQ887gF/uljcChyc970X2/K+Bq4Cnz7D+BuCLQIBNwOOjnsNKPnMY5CM5tgC7uuXPA9ckyRjnOEoL9ltVj1bVq93bA/R+n2Q5G/RjV+4C7gb+fpyTO0cG6fmDwCer6mWAqnppzHMctUF6LuDN3fKPAn8zxvmNXFU9Bhx/nSFbgAeq5wBwUZJLRjmHlRwOg3wkx2tjquok8Arw1rHMbvTO9iNIbqX3k8dytmDP3en2uqr6wjgndg4N8uf8E8BPJPmfSQ4k2Ty22Z0bg/T8YeDnkxyh99Tjr4xnahNzzj9yaMk8yqrxSfLzwDTwbyY9l3MpyQ8BHwfeP+GpjNsaepeWZuidHT6W5Iqq+tZEZ3Vu3QzcX1X3JPlp4L8meVtV/d9JT2y5WslnDoN8JMdrY5KsoXc6+s2xzG70BvoIkiT/Fvgt4N1V9d0xze1cWajnNwFvA2aTHKZ3bXbPMr8pPcif8xFgT1X9Q1V9Dfjf9MJiuRqk51uBhwGq6s+Af0Tvc5dWqoH+e1+MlRwOg3wkxx5ga7f8HuB/VHe3ZxlasN8kbwf+C71gWO7XoWGBnqvqlapaW1Xrq2o9vfss766qg5OZ7kgM8vf6v9E7ayDJWnqXmZ4f5yRHbJCe/xq4BiDJv6AXDv9nrLMcrz3ALd1TS5uAV6rq2CgPsGIvK9UZPpIjyUeAg1W1B/g0vdPPOXo3f26a3IwXZ8B+/zPwRuD3u/vuf11V757YpBdpwJ5XlAF73gdcm+RZ4PvAf6qq5XpGPGjPdwCfSvIf6N2cfv8y/kGPJJ+lF/Bru/sodwJvAKiq36N3X+UGYA54FfjAyOewjL9/kqRzZCVfVpIkDclwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1DAdJUsNwkCQ1/h9hxxDL3acZQAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y.hist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,random_state=101)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense,Activation\n",
    "from tensorflow.keras.optimizers import SGD,Adam\n",
    "from tensorflow.keras.utils import to_categorical\n",
    "from tensorflow import keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_15 (Dense)             (None, 10)                140       \n",
      "_________________________________________________________________\n",
      "dense_16 (Dense)             (None, 2)                 22        \n",
      "=================================================================\n",
      "Total params: 162\n",
      "Trainable params: 162\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "#model.add(Dense(2,input_shape = (X_train.shape[1],), activation = 'softmax'))\n",
    "model.add(Dense(10,input_shape = (X_train.shape[1],), activation = 'relu'))\n",
    "#model.add(keras.layers.Dropout(0.5))\n",
    "#model.add(Dense(10,activation = 'relu'))\n",
    "#model.add(keras.layers.Dropout(0.5))\n",
    "#model.add(Dense(10,activation = 'relu'))\n",
    "#model.add(keras.layers.Dropout(0.5))\n",
    "model.add(Dense(2, activation = 'softmax'))\n",
    "model.summary()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [],
   "source": [
    "adam = Adam(0.001)\n",
    "#sgd = SGD(0.0001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_binary_train= to_categorical(y_train)\n",
    "y_binary_test = to_categorical(y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss = 'categorical_crossentropy', optimizer = adam, metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 22792 samples, validate on 9769 samples\n",
      "Epoch 1/200\n",
      "22792/22792 [==============================] - 4s 164us/step - loss: 2.1302 - acc: 0.6726 - val_loss: 1.7056 - val_acc: 0.7380\n",
      "Epoch 2/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.6098 - acc: 0.7500 - val_loss: 1.6485 - val_acc: 0.7569\n",
      "Epoch 3/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.5761 - acc: 0.7563 - val_loss: 1.6261 - val_acc: 0.7592\n",
      "Epoch 4/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.5582 - acc: 0.7570 - val_loss: 1.6156 - val_acc: 0.7605\n",
      "Epoch 5/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.5465 - acc: 0.7578 - val_loss: 1.6042 - val_acc: 0.7605\n",
      "Epoch 6/200\n",
      "22792/22792 [==============================] - 3s 138us/step - loss: 1.5389 - acc: 0.7577 - val_loss: 1.5990 - val_acc: 0.7606\n",
      "Epoch 7/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.5324 - acc: 0.7579 - val_loss: 1.5943 - val_acc: 0.7608\n",
      "Epoch 8/200\n",
      "22792/22792 [==============================] - 2s 100us/step - loss: 1.5268 - acc: 0.7604 - val_loss: 1.5857 - val_acc: 0.7698\n",
      "Epoch 9/200\n",
      "22792/22792 [==============================] - 2s 109us/step - loss: 1.5205 - acc: 0.7678 - val_loss: 1.5801 - val_acc: 0.7724\n",
      "Epoch 10/200\n",
      "22792/22792 [==============================] - 2s 104us/step - loss: 1.5144 - acc: 0.7722 - val_loss: 1.5732 - val_acc: 0.7750\n",
      "Epoch 11/200\n",
      "22792/22792 [==============================] - 2s 102us/step - loss: 1.5086 - acc: 0.7758 - val_loss: 1.5662 - val_acc: 0.7798\n",
      "Epoch 12/200\n",
      "22792/22792 [==============================] - 2s 104us/step - loss: 1.5034 - acc: 0.7796 - val_loss: 1.5609 - val_acc: 0.7787\n",
      "Epoch 13/200\n",
      "22792/22792 [==============================] - 2s 105us/step - loss: 1.5013 - acc: 0.7799 - val_loss: 1.5563 - val_acc: 0.7833\n",
      "Epoch 14/200\n",
      "22792/22792 [==============================] - 2s 105us/step - loss: 1.4976 - acc: 0.7822 - val_loss: 1.5639 - val_acc: 0.7784\n",
      "Epoch 15/200\n",
      "22792/22792 [==============================] - 2s 104us/step - loss: 1.4951 - acc: 0.7831 - val_loss: 1.5528 - val_acc: 0.7817\n",
      "Epoch 16/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4937 - acc: 0.7830 - val_loss: 1.5538 - val_acc: 0.7808\n",
      "Epoch 17/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4928 - acc: 0.7844 - val_loss: 1.5533 - val_acc: 0.7842\n",
      "Epoch 18/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4911 - acc: 0.7849 - val_loss: 1.5511 - val_acc: 0.7912\n",
      "Epoch 19/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4900 - acc: 0.7872 - val_loss: 1.5612 - val_acc: 0.7813\n",
      "Epoch 20/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4895 - acc: 0.7865 - val_loss: 1.5459 - val_acc: 0.7873\n",
      "Epoch 21/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4889 - acc: 0.7853 - val_loss: 1.5483 - val_acc: 0.7878\n",
      "Epoch 22/200\n",
      "22792/22792 [==============================] - 3s 117us/step - loss: 1.4880 - acc: 0.7875 - val_loss: 1.5450 - val_acc: 0.7901\n",
      "Epoch 23/200\n",
      "22792/22792 [==============================] - 3s 116us/step - loss: 1.4885 - acc: 0.7874 - val_loss: 1.5442 - val_acc: 0.7867\n",
      "Epoch 24/200\n",
      "22792/22792 [==============================] - 3s 117us/step - loss: 1.4868 - acc: 0.7874 - val_loss: 1.5483 - val_acc: 0.7852\n",
      "Epoch 25/200\n",
      "22792/22792 [==============================] - 3s 142us/step - loss: 1.4866 - acc: 0.7882 - val_loss: 1.5435 - val_acc: 0.7913\n",
      "Epoch 26/200\n",
      "22792/22792 [==============================] - 3s 113us/step - loss: 1.4867 - acc: 0.7883 - val_loss: 1.5478 - val_acc: 0.7908\n",
      "Epoch 27/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4869 - acc: 0.7884 - val_loss: 1.5423 - val_acc: 0.7927\n",
      "Epoch 28/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4869 - acc: 0.7871 - val_loss: 1.5461 - val_acc: 0.7868\n",
      "Epoch 29/200\n",
      "22792/22792 [==============================] - 2s 101us/step - loss: 1.4865 - acc: 0.7860 - val_loss: 1.5421 - val_acc: 0.7896\n",
      "Epoch 30/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4864 - acc: 0.7893 - val_loss: 1.5432 - val_acc: 0.7889\n",
      "Epoch 31/200\n",
      "22792/22792 [==============================] - 2s 88us/step - loss: 1.4852 - acc: 0.7862 - val_loss: 1.5482 - val_acc: 0.7886\n",
      "Epoch 32/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4858 - acc: 0.7877 - val_loss: 1.5420 - val_acc: 0.7927\n",
      "Epoch 33/200\n",
      "22792/22792 [==============================] - 2s 100us/step - loss: 1.4848 - acc: 0.7894 - val_loss: 1.5423 - val_acc: 0.7898\n",
      "Epoch 34/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4857 - acc: 0.7880 - val_loss: 1.5408 - val_acc: 0.7912\n",
      "Epoch 35/200\n",
      "22792/22792 [==============================] - 3s 114us/step - loss: 1.4850 - acc: 0.7876 - val_loss: 1.5430 - val_acc: 0.7883\n",
      "Epoch 36/200\n",
      "22792/22792 [==============================] - 2s 110us/step - loss: 1.4859 - acc: 0.7869 - val_loss: 1.5453 - val_acc: 0.7915\n",
      "Epoch 37/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.4858 - acc: 0.7888 - val_loss: 1.5414 - val_acc: 0.7930\n",
      "Epoch 38/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4859 - acc: 0.7883 - val_loss: 1.5429 - val_acc: 0.7888\n",
      "Epoch 39/200\n",
      "22792/22792 [==============================] - 2s 88us/step - loss: 1.4849 - acc: 0.7875 - val_loss: 1.5444 - val_acc: 0.7898\n",
      "Epoch 40/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4843 - acc: 0.7895 - val_loss: 1.5429 - val_acc: 0.7922\n",
      "Epoch 41/200\n",
      "22792/22792 [==============================] - 2s 108us/step - loss: 1.4841 - acc: 0.7886 - val_loss: 1.5418 - val_acc: 0.7888\n",
      "Epoch 42/200\n",
      "22792/22792 [==============================] - 2s 101us/step - loss: 1.4846 - acc: 0.7886 - val_loss: 1.5417 - val_acc: 0.7905\n",
      "Epoch 43/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4845 - acc: 0.7883 - val_loss: 1.5417 - val_acc: 0.7890\n",
      "Epoch 44/200\n",
      "22792/22792 [==============================] - 3s 127us/step - loss: 1.4844 - acc: 0.7880 - val_loss: 1.5427 - val_acc: 0.7911\n",
      "Epoch 45/200\n",
      "22792/22792 [==============================] - 4s 158us/step - loss: 1.4842 - acc: 0.7871 - val_loss: 1.5434 - val_acc: 0.7907\n",
      "Epoch 46/200\n",
      "22792/22792 [==============================] - 3s 147us/step - loss: 1.4850 - acc: 0.7882 - val_loss: 1.5419 - val_acc: 0.7871\n",
      "Epoch 47/200\n",
      "22792/22792 [==============================] - 3s 145us/step - loss: 1.4847 - acc: 0.7893 - val_loss: 1.5419 - val_acc: 0.7906\n",
      "Epoch 48/200\n",
      "22792/22792 [==============================] - 2s 102us/step - loss: 1.4842 - acc: 0.7899 - val_loss: 1.5404 - val_acc: 0.7903\n",
      "Epoch 49/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4845 - acc: 0.7897 - val_loss: 1.5415 - val_acc: 0.7890\n",
      "Epoch 50/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4847 - acc: 0.7898 - val_loss: 1.5436 - val_acc: 0.7873\n",
      "Epoch 51/200\n",
      "22792/22792 [==============================] - 3s 119us/step - loss: 1.4846 - acc: 0.7885 - val_loss: 1.5443 - val_acc: 0.7880\n",
      "Epoch 52/200\n",
      "22792/22792 [==============================] - 3s 138us/step - loss: 1.4842 - acc: 0.7892 - val_loss: 1.5407 - val_acc: 0.7906\n",
      "Epoch 53/200\n",
      "22792/22792 [==============================] - 3s 140us/step - loss: 1.4844 - acc: 0.7894 - val_loss: 1.5444 - val_acc: 0.7850\n",
      "Epoch 54/200\n",
      "22792/22792 [==============================] - 3s 138us/step - loss: 1.4837 - acc: 0.7901 - val_loss: 1.5413 - val_acc: 0.7910\n",
      "Epoch 55/200\n",
      "22792/22792 [==============================] - 3s 149us/step - loss: 1.4837 - acc: 0.7880 - val_loss: 1.5418 - val_acc: 0.7866\n",
      "Epoch 56/200\n",
      "22792/22792 [==============================] - 3s 133us/step - loss: 1.4840 - acc: 0.7894 - val_loss: 1.5404 - val_acc: 0.7885\n",
      "Epoch 57/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4844 - acc: 0.7876 - val_loss: 1.5429 - val_acc: 0.7901\n",
      "Epoch 58/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4843 - acc: 0.7869 - val_loss: 1.5430 - val_acc: 0.7880\n",
      "Epoch 59/200\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4847 - acc: 0.7883 - val_loss: 1.5424 - val_acc: 0.7906\n",
      "Epoch 60/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4844 - acc: 0.7888 - val_loss: 1.5410 - val_acc: 0.7893\n",
      "Epoch 61/200\n",
      "22792/22792 [==============================] - 2s 90us/step - loss: 1.4835 - acc: 0.7886 - val_loss: 1.5456 - val_acc: 0.7855\n",
      "Epoch 62/200\n",
      "22792/22792 [==============================] - 2s 87us/step - loss: 1.4840 - acc: 0.7890 - val_loss: 1.5417 - val_acc: 0.7905\n",
      "Epoch 63/200\n",
      "22792/22792 [==============================] - 2s 90us/step - loss: 1.4832 - acc: 0.7898 - val_loss: 1.5427 - val_acc: 0.7941\n",
      "Epoch 64/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4845 - acc: 0.7878 - val_loss: 1.5533 - val_acc: 0.7813\n",
      "Epoch 65/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4837 - acc: 0.7872 - val_loss: 1.5444 - val_acc: 0.7879\n",
      "Epoch 66/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4839 - acc: 0.7902 - val_loss: 1.5422 - val_acc: 0.7879\n",
      "Epoch 67/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4829 - acc: 0.7906 - val_loss: 1.5418 - val_acc: 0.7874\n",
      "Epoch 68/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4831 - acc: 0.7890 - val_loss: 1.5414 - val_acc: 0.7948\n",
      "Epoch 69/200\n",
      "22792/22792 [==============================] - 2s 89us/step - loss: 1.4837 - acc: 0.7894 - val_loss: 1.5432 - val_acc: 0.7907\n",
      "Epoch 70/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4838 - acc: 0.7874 - val_loss: 1.5428 - val_acc: 0.7870\n",
      "Epoch 71/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4833 - acc: 0.7898 - val_loss: 1.5476 - val_acc: 0.7827\n",
      "Epoch 72/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4833 - acc: 0.7875 - val_loss: 1.5408 - val_acc: 0.7885\n",
      "Epoch 73/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4836 - acc: 0.7876 - val_loss: 1.5415 - val_acc: 0.7943\n",
      "Epoch 74/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4827 - acc: 0.7887 - val_loss: 1.5418 - val_acc: 0.7885\n",
      "Epoch 75/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4827 - acc: 0.7900 - val_loss: 1.5411 - val_acc: 0.7941\n",
      "Epoch 76/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4830 - acc: 0.7889 - val_loss: 1.5405 - val_acc: 0.7877\n",
      "Epoch 77/200\n",
      "22792/22792 [==============================] - 2s 90us/step - loss: 1.4824 - acc: 0.7881 - val_loss: 1.5426 - val_acc: 0.7889\n",
      "Epoch 78/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4835 - acc: 0.7890 - val_loss: 1.5420 - val_acc: 0.7878\n",
      "Epoch 79/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4828 - acc: 0.7894 - val_loss: 1.5421 - val_acc: 0.7868\n",
      "Epoch 80/200\n",
      "22792/22792 [==============================] - 2s 103us/step - loss: 1.4833 - acc: 0.7883 - val_loss: 1.5397 - val_acc: 0.7928\n",
      "Epoch 81/200\n",
      "22792/22792 [==============================] - 3s 116us/step - loss: 1.4822 - acc: 0.7921 - val_loss: 1.5422 - val_acc: 0.7877\n",
      "Epoch 82/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4838 - acc: 0.7887 - val_loss: 1.5404 - val_acc: 0.7869\n",
      "Epoch 83/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4822 - acc: 0.7920 - val_loss: 1.5433 - val_acc: 0.7853\n",
      "Epoch 84/200\n",
      "22792/22792 [==============================] - 2s 90us/step - loss: 1.4830 - acc: 0.7884 - val_loss: 1.5474 - val_acc: 0.7840\n",
      "Epoch 85/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4821 - acc: 0.7891 - val_loss: 1.5458 - val_acc: 0.7878\n",
      "Epoch 86/200\n",
      "22792/22792 [==============================] - 2s 105us/step - loss: 1.4830 - acc: 0.7901 - val_loss: 1.5403 - val_acc: 0.7936\n",
      "Epoch 87/200\n",
      "22792/22792 [==============================] - 2s 102us/step - loss: 1.4830 - acc: 0.7890 - val_loss: 1.5399 - val_acc: 0.7924\n",
      "Epoch 88/200\n",
      "22792/22792 [==============================] - 2s 101us/step - loss: 1.4837 - acc: 0.7890 - val_loss: 1.5408 - val_acc: 0.7875\n",
      "Epoch 89/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4827 - acc: 0.7896 - val_loss: 1.5419 - val_acc: 0.7889\n",
      "Epoch 90/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4827 - acc: 0.7898 - val_loss: 1.5405 - val_acc: 0.7912\n",
      "Epoch 91/200\n",
      "22792/22792 [==============================] - 3s 112us/step - loss: 1.4831 - acc: 0.7888 - val_loss: 1.5421 - val_acc: 0.7868\n",
      "Epoch 92/200\n",
      "22792/22792 [==============================] - 3s 113us/step - loss: 1.4837 - acc: 0.7880 - val_loss: 1.5401 - val_acc: 0.7895\n",
      "Epoch 93/200\n",
      "22792/22792 [==============================] - 3s 112us/step - loss: 1.4824 - acc: 0.7890 - val_loss: 1.5391 - val_acc: 0.7934\n",
      "Epoch 94/200\n",
      "22792/22792 [==============================] - 2s 102us/step - loss: 1.4827 - acc: 0.7888 - val_loss: 1.5402 - val_acc: 0.7902\n",
      "Epoch 95/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4823 - acc: 0.7901 - val_loss: 1.5446 - val_acc: 0.7840\n",
      "Epoch 96/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4820 - acc: 0.7898 - val_loss: 1.5395 - val_acc: 0.7920\n",
      "Epoch 97/200\n",
      "22792/22792 [==============================] - 2s 90us/step - loss: 1.4822 - acc: 0.7887 - val_loss: 1.5428 - val_acc: 0.7873\n",
      "Epoch 98/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4814 - acc: 0.7899 - val_loss: 1.5401 - val_acc: 0.7903\n",
      "Epoch 99/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4821 - acc: 0.7905 - val_loss: 1.5415 - val_acc: 0.7942\n",
      "Epoch 100/200\n",
      "22792/22792 [==============================] - 2s 90us/step - loss: 1.4826 - acc: 0.7900 - val_loss: 1.5391 - val_acc: 0.7911\n",
      "Epoch 101/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4824 - acc: 0.7884 - val_loss: 1.5399 - val_acc: 0.7909\n",
      "Epoch 102/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4825 - acc: 0.7895 - val_loss: 1.5409 - val_acc: 0.7914\n",
      "Epoch 103/200\n",
      "22792/22792 [==============================] - 2s 88us/step - loss: 1.4823 - acc: 0.7919 - val_loss: 1.5415 - val_acc: 0.7920\n",
      "Epoch 104/200\n",
      "22792/22792 [==============================] - 2s 89us/step - loss: 1.4815 - acc: 0.7909 - val_loss: 1.5398 - val_acc: 0.7903\n",
      "Epoch 105/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4821 - acc: 0.7892 - val_loss: 1.5386 - val_acc: 0.7917\n",
      "Epoch 106/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.4817 - acc: 0.7909 - val_loss: 1.5390 - val_acc: 0.7904\n",
      "Epoch 107/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4821 - acc: 0.7910 - val_loss: 1.5400 - val_acc: 0.7919\n",
      "Epoch 108/200\n",
      "22792/22792 [==============================] - 2s 104us/step - loss: 1.4821 - acc: 0.7897 - val_loss: 1.5384 - val_acc: 0.7922\n",
      "Epoch 109/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4825 - acc: 0.7882 - val_loss: 1.5417 - val_acc: 0.7924\n",
      "Epoch 110/200\n",
      "22792/22792 [==============================] - 3s 117us/step - loss: 1.4821 - acc: 0.7917 - val_loss: 1.5395 - val_acc: 0.7915\n",
      "Epoch 111/200\n",
      "22792/22792 [==============================] - 2s 103us/step - loss: 1.4818 - acc: 0.7887 - val_loss: 1.5391 - val_acc: 0.7913\n",
      "Epoch 112/200\n",
      "22792/22792 [==============================] - 3s 113us/step - loss: 1.4818 - acc: 0.7892 - val_loss: 1.5433 - val_acc: 0.7872\n",
      "Epoch 113/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4814 - acc: 0.7894 - val_loss: 1.5402 - val_acc: 0.7893\n",
      "Epoch 114/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4818 - acc: 0.7896 - val_loss: 1.5407 - val_acc: 0.7921\n",
      "Epoch 115/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4813 - acc: 0.7911 - val_loss: 1.5402 - val_acc: 0.7887\n",
      "Epoch 116/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4817 - acc: 0.7895 - val_loss: 1.5384 - val_acc: 0.7920\n",
      "Epoch 117/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4820 - acc: 0.7882 - val_loss: 1.5409 - val_acc: 0.7909\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 118/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4812 - acc: 0.7914 - val_loss: 1.5415 - val_acc: 0.7895\n",
      "Epoch 119/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4818 - acc: 0.7912 - val_loss: 1.5407 - val_acc: 0.7897\n",
      "Epoch 120/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4815 - acc: 0.7898 - val_loss: 1.5405 - val_acc: 0.7887\n",
      "Epoch 121/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4814 - acc: 0.7909 - val_loss: 1.5387 - val_acc: 0.7942\n",
      "Epoch 122/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4816 - acc: 0.7897 - val_loss: 1.5400 - val_acc: 0.7888\n",
      "Epoch 123/200\n",
      "22792/22792 [==============================] - 2s 102us/step - loss: 1.4819 - acc: 0.7903 - val_loss: 1.5427 - val_acc: 0.7895\n",
      "Epoch 124/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4816 - acc: 0.7894 - val_loss: 1.5404 - val_acc: 0.7905\n",
      "Epoch 125/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4819 - acc: 0.7891 - val_loss: 1.5399 - val_acc: 0.7913\n",
      "Epoch 126/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4812 - acc: 0.7901 - val_loss: 1.5402 - val_acc: 0.7922\n",
      "Epoch 127/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4819 - acc: 0.7895 - val_loss: 1.5393 - val_acc: 0.7913\n",
      "Epoch 128/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4819 - acc: 0.7899 - val_loss: 1.5449 - val_acc: 0.7870\n",
      "Epoch 129/200\n",
      "22792/22792 [==============================] - 2s 108us/step - loss: 1.4821 - acc: 0.7873 - val_loss: 1.5412 - val_acc: 0.7891\n",
      "Epoch 130/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4810 - acc: 0.7916 - val_loss: 1.5421 - val_acc: 0.7893\n",
      "Epoch 131/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4809 - acc: 0.7893 - val_loss: 1.5399 - val_acc: 0.7928\n",
      "Epoch 132/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4816 - acc: 0.7889 - val_loss: 1.5422 - val_acc: 0.7907\n",
      "Epoch 133/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4818 - acc: 0.7894 - val_loss: 1.5414 - val_acc: 0.7911\n",
      "Epoch 134/200\n",
      "22792/22792 [==============================] - 2s 100us/step - loss: 1.4818 - acc: 0.7889 - val_loss: 1.5451 - val_acc: 0.7869\n",
      "Epoch 135/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4813 - acc: 0.7892 - val_loss: 1.5387 - val_acc: 0.7919\n",
      "Epoch 136/200\n",
      "22792/22792 [==============================] - 2s 101us/step - loss: 1.4813 - acc: 0.7912 - val_loss: 1.5423 - val_acc: 0.7893\n",
      "Epoch 137/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4813 - acc: 0.7897 - val_loss: 1.5428 - val_acc: 0.7866\n",
      "Epoch 138/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4810 - acc: 0.7897 - val_loss: 1.5511 - val_acc: 0.7739\n",
      "Epoch 139/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.4809 - acc: 0.7916 - val_loss: 1.5409 - val_acc: 0.7895\n",
      "Epoch 140/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4813 - acc: 0.7890 - val_loss: 1.5390 - val_acc: 0.7928\n",
      "Epoch 141/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4806 - acc: 0.7898 - val_loss: 1.5381 - val_acc: 0.7938\n",
      "Epoch 142/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4806 - acc: 0.7893 - val_loss: 1.5390 - val_acc: 0.7932\n",
      "Epoch 143/200\n",
      "22792/22792 [==============================] - 2s 87us/step - loss: 1.4809 - acc: 0.7914 - val_loss: 1.5380 - val_acc: 0.7916\n",
      "Epoch 144/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4807 - acc: 0.7911 - val_loss: 1.5385 - val_acc: 0.7920\n",
      "Epoch 145/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4812 - acc: 0.7904 - val_loss: 1.5382 - val_acc: 0.7914\n",
      "Epoch 146/200\n",
      "22792/22792 [==============================] - 2s 101us/step - loss: 1.4810 - acc: 0.7891 - val_loss: 1.5385 - val_acc: 0.7927\n",
      "Epoch 147/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4807 - acc: 0.7900 - val_loss: 1.5398 - val_acc: 0.7926\n",
      "Epoch 148/200\n",
      "22792/22792 [==============================] - 2s 105us/step - loss: 1.4812 - acc: 0.7912 - val_loss: 1.5379 - val_acc: 0.7946\n",
      "Epoch 149/200\n",
      "22792/22792 [==============================] - 2s 102us/step - loss: 1.4811 - acc: 0.7893 - val_loss: 1.5394 - val_acc: 0.7905\n",
      "Epoch 150/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4807 - acc: 0.7897 - val_loss: 1.5393 - val_acc: 0.7919\n",
      "Epoch 151/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4805 - acc: 0.7929 - val_loss: 1.5408 - val_acc: 0.7917\n",
      "Epoch 152/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4811 - acc: 0.7892 - val_loss: 1.5398 - val_acc: 0.7929\n",
      "Epoch 153/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4811 - acc: 0.7892 - val_loss: 1.5400 - val_acc: 0.7902\n",
      "Epoch 154/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.4804 - acc: 0.7914 - val_loss: 1.5394 - val_acc: 0.7933\n",
      "Epoch 155/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4809 - acc: 0.7909 - val_loss: 1.5402 - val_acc: 0.7889\n",
      "Epoch 156/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4808 - acc: 0.7913 - val_loss: 1.5384 - val_acc: 0.7936\n",
      "Epoch 157/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4813 - acc: 0.7899 - val_loss: 1.5388 - val_acc: 0.7908\n",
      "Epoch 158/200\n",
      "22792/22792 [==============================] - 2s 101us/step - loss: 1.4810 - acc: 0.7905 - val_loss: 1.5424 - val_acc: 0.7935\n",
      "Epoch 159/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4811 - acc: 0.7900 - val_loss: 1.5419 - val_acc: 0.7903\n",
      "Epoch 160/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4801 - acc: 0.7907 - val_loss: 1.5396 - val_acc: 0.7907\n",
      "Epoch 161/200\n",
      "22792/22792 [==============================] - 3s 113us/step - loss: 1.4806 - acc: 0.7899 - val_loss: 1.5389 - val_acc: 0.7909\n",
      "Epoch 162/200\n",
      "22792/22792 [==============================] - 2s 103us/step - loss: 1.4800 - acc: 0.7914 - val_loss: 1.5390 - val_acc: 0.7919\n",
      "Epoch 163/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4808 - acc: 0.7890 - val_loss: 1.5395 - val_acc: 0.7913\n",
      "Epoch 164/200\n",
      "22792/22792 [==============================] - 3s 118us/step - loss: 1.4811 - acc: 0.7900 - val_loss: 1.5379 - val_acc: 0.7922\n",
      "Epoch 165/200\n",
      "22792/22792 [==============================] - 2s 108us/step - loss: 1.4807 - acc: 0.7891 - val_loss: 1.5399 - val_acc: 0.7950\n",
      "Epoch 166/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4805 - acc: 0.7908 - val_loss: 1.5410 - val_acc: 0.7901\n",
      "Epoch 167/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4801 - acc: 0.7906 - val_loss: 1.5380 - val_acc: 0.7923\n",
      "Epoch 168/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4805 - acc: 0.7902 - val_loss: 1.5501 - val_acc: 0.7804\n",
      "Epoch 169/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4807 - acc: 0.7899 - val_loss: 1.5380 - val_acc: 0.7925\n",
      "Epoch 170/200\n",
      "22792/22792 [==============================] - 2s 101us/step - loss: 1.4804 - acc: 0.7911 - val_loss: 1.5400 - val_acc: 0.7924\n",
      "Epoch 171/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4797 - acc: 0.7908 - val_loss: 1.5403 - val_acc: 0.7912\n",
      "Epoch 172/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4803 - acc: 0.7919 - val_loss: 1.5379 - val_acc: 0.7933\n",
      "Epoch 173/200\n",
      "22792/22792 [==============================] - 2s 102us/step - loss: 1.4801 - acc: 0.7912 - val_loss: 1.5414 - val_acc: 0.7872\n",
      "Epoch 174/200\n",
      "22792/22792 [==============================] - 2s 104us/step - loss: 1.4809 - acc: 0.7896 - val_loss: 1.5393 - val_acc: 0.7937\n",
      "Epoch 175/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4797 - acc: 0.7908 - val_loss: 1.5381 - val_acc: 0.7930\n",
      "Epoch 176/200\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4809 - acc: 0.7918 - val_loss: 1.5390 - val_acc: 0.7916\n",
      "Epoch 177/200\n",
      "22792/22792 [==============================] - 2s 96us/step - loss: 1.4800 - acc: 0.7911 - val_loss: 1.5388 - val_acc: 0.7923\n",
      "Epoch 178/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.4806 - acc: 0.7915 - val_loss: 1.5389 - val_acc: 0.7917\n",
      "Epoch 179/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.4806 - acc: 0.7904 - val_loss: 1.5377 - val_acc: 0.7954\n",
      "Epoch 180/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4805 - acc: 0.7892 - val_loss: 1.5379 - val_acc: 0.7925\n",
      "Epoch 181/200\n",
      "22792/22792 [==============================] - 2s 100us/step - loss: 1.4802 - acc: 0.7915 - val_loss: 1.5394 - val_acc: 0.7921\n",
      "Epoch 182/200\n",
      "22792/22792 [==============================] - 2s 104us/step - loss: 1.4802 - acc: 0.7900 - val_loss: 1.5382 - val_acc: 0.7924\n",
      "Epoch 183/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4801 - acc: 0.7912 - val_loss: 1.5386 - val_acc: 0.7923\n",
      "Epoch 184/200\n",
      "22792/22792 [==============================] - 2s 93us/step - loss: 1.4797 - acc: 0.7895 - val_loss: 1.5420 - val_acc: 0.7913\n",
      "Epoch 185/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4799 - acc: 0.7907 - val_loss: 1.5434 - val_acc: 0.7842\n",
      "Epoch 186/200\n",
      "22792/22792 [==============================] - 2s 98us/step - loss: 1.4804 - acc: 0.7884 - val_loss: 1.5426 - val_acc: 0.7894\n",
      "Epoch 187/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4799 - acc: 0.7904 - val_loss: 1.5386 - val_acc: 0.7918\n",
      "Epoch 188/200\n",
      "22792/22792 [==============================] - 2s 94us/step - loss: 1.4806 - acc: 0.7894 - val_loss: 1.5394 - val_acc: 0.7888\n",
      "Epoch 189/200\n",
      "22792/22792 [==============================] - 2s 99us/step - loss: 1.4804 - acc: 0.7910 - val_loss: 1.5403 - val_acc: 0.7872\n",
      "Epoch 190/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4796 - acc: 0.7907 - val_loss: 1.5394 - val_acc: 0.7903\n",
      "Epoch 191/200\n",
      "22792/22792 [==============================] - 2s 97us/step - loss: 1.4806 - acc: 0.7894 - val_loss: 1.5383 - val_acc: 0.7923\n",
      "Epoch 192/200\n",
      "22792/22792 [==============================] - 3s 121us/step - loss: 1.4799 - acc: 0.7898 - val_loss: 1.5433 - val_acc: 0.7869\n",
      "Epoch 193/200\n",
      "22792/22792 [==============================] - 3s 113us/step - loss: 1.4804 - acc: 0.7902 - val_loss: 1.5383 - val_acc: 0.7946\n",
      "Epoch 194/200\n",
      "22792/22792 [==============================] - 2s 92us/step - loss: 1.4807 - acc: 0.7914 - val_loss: 1.5403 - val_acc: 0.7903\n",
      "Epoch 195/200\n",
      "22792/22792 [==============================] - 2s 91us/step - loss: 1.4810 - acc: 0.7890 - val_loss: 1.5415 - val_acc: 0.7890\n",
      "Epoch 196/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4797 - acc: 0.7933 - val_loss: 1.5468 - val_acc: 0.7921\n",
      "Epoch 197/200\n",
      "22792/22792 [==============================] - 3s 128us/step - loss: 1.4803 - acc: 0.7906 - val_loss: 1.5408 - val_acc: 0.7915\n",
      "Epoch 198/200\n",
      "22792/22792 [==============================] - 2s 95us/step - loss: 1.4804 - acc: 0.7918 - val_loss: 1.5400 - val_acc: 0.7902\n",
      "Epoch 199/200\n",
      "22792/22792 [==============================] - 3s 116us/step - loss: 1.4799 - acc: 0.7916 - val_loss: 1.5390 - val_acc: 0.7957\n",
      "Epoch 200/200\n",
      "22792/22792 [==============================] - 3s 111us/step - loss: 1.4799 - acc: 0.7905 - val_loss: 1.5384 - val_acc: 0.7907\n"
     ]
    }
   ],
   "source": [
    "H = model.fit(X_train, y_binary_train, \n",
    "              validation_data=(X_test, y_binary_test),\n",
    "              epochs = 200,batch_size=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1a410c08d0>]"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztnXl4VNX5xz8nGwQCIZAEAiELm+yyIyIKCBaXilqrWFe0Lq36c7dqq3WtrdrSutUdl7birmhdQASkgAoRZF8TCEmALCSQhOxzfn+cuWQymTUzWZi8n+fJMzP3nnvvuZk73/ve97zve5TWGkEQBKF9ENbaHRAEQRBaDhF9QRCEdoSIviAIQjtCRF8QBKEdIaIvCILQjhDRFwRBaEeI6AuCILQjRPQFQRDaESL6giAI7YiI1u6AM/Hx8TotLa21uyEIgnBckZGRUai1TvDWrs2JflpaGmvXrm3tbgiCIBxXKKX2+tJO3DuCIAjtCBF9QRCEdoSIviAIQjtCRF8QBKEdIaIvCILQjhDRFwRBaEeI6AuCILQjRPQFQWg+KivhpZegpqa1eyLYEdEXmgebTX7oArz8Mlx/PSxa1No9EeyI6AvNw9y5MHEiVFW1dk+E1kJreP55837z5tbti3AMEX0h+GgNX3wB69bBo4+2dm+E1mLZMti2zbzfsqVVuyLUI6IvBJ/du6GgAHr1gscfhx9/bO0eCa3B889D9+5wyili6bchRPSF4LNypXl9911ITDSunurq1u2T0LLk5cHHH8PVV8PYsbB1qxnnaevs3Anffw91da3dk2ZDRF8IPqtWQWwsTJ4ML7wAGzYYi7+tsm4dTJ0Kmza1dk9alkOH4Mwz4Ztvgr/vV16B2lq44QYYOhTKy2HfvuAfJ5gUFJhr9qSToHdvc8P6+GPT9xBCRF8IPitXwqRJEBYG554Lv/qV8e3/9FNr96wxe/bAWWfB8uVw661mPKK98Je/wJdfmiexsrLg7bemBl58EWbNgv79jehD23fx/N//QUkJPP00nH46fPghnH8+9OhhrpH//re1exgURPSF4FJSYn7cJ59cv+zpp41vd+7cthXGeeiQ+TFXVsKNN8KSJUYE2wO5ueZ7mTgRsrPhgQeCt+9PPzXund/8xny2RL8tD+Z+9BEsWAD33w833wz/+Y+x/L/5xpzHpk1wwQWhYfVrrdvU39ixY7XQzNhsWh892jz7/uILrUHrJUsaLv/gA7P8kUea57j+UlGh9ZQpWkdFab1smdZVVVr376/18OFa19a2du+an+uv1zoyUuvdu7W+4Qatw8K0Xrs2OPs+/XStU1Ia/h979dJ67tzg7D/YFBZq3bOn1qNGaV1d7bqNdV0vWtSyffMDYK32QWPF0m9v5ObCaadBTAxMmQJPPGHC6oLl1li5EsLDYcKEhssvuAAuuggefrj1fec2G1x5JaxYAW+8Yf4fUVHw5z+bvr3+euv2r7nZudP43K+7Dvr1M+MtiYlw7bXGDx8I27aZJ6brrzfXgcXQoS3r3tHauOzuvtv7cW+9FYqKYP58iIx03eaUUyAiApYuDX5fWxpf7gwt+SeWfjPy1Vdax8dr3bmz1jffrPXo0cZ6Aa0HDND69tu1/u67wI4xfbrWY8a4Xpefb44/bpzWdXWBHScQ7rjDnPOTTzZcbrNpPWmS1klJWpeVtU7fWoI5c7Tu1Enr/fvrl733nvmf/PWvge37llvME8SBAw2X33yz1jEx5n/cnNTUaL1ggbnGrGu7Uyet33jDdfuFC02bBx7wvu9Jk7Q+6aTg9jeI4KOl3+oi7/wnot8M1NZq/Yc/aK2UcV9s3Vq/Ljtb6+ef13rWLOPqCAvTev36ph2npsbcUG66yX2b+fPNZff55007RqD8/e/m+Dff7FqAVq406x96qOX71hKsW2fO7777Gi632bQ+5xwjkFlZTdt3WZnWsbFaX3JJ43X//Kc5bnZ20/btjSNHzHebmmqOM2iQ1i++aNxXp51mll1zTUO35qFD5gY/YoRx73njvvu0Dg83xwomNpsxiFas0Hrp0ibvJqiiD8wCtgO7gHtcrJ8HrLf/7QBKHNb9Bdhk/7vY27FE9IPM/v1aT5tmvuq5c7UuL3ffNi/PiL4vVo8rMjLMcd5+232bqirzQ5s5s2nH8IVnn9V66NDGf0OGmBvf+ed79ttfcIG5eTlawm2JXbu03rmzadueeabWcXFaFxc3Xrd3rznvs85qmkX+yivm+1+xovG65cvNui++8H+/3vjpJ627dTP7nzJF608+afgkWVOj9e9/b9aPGKH1tm1m+VVXGRHPyPDtOIsXm338979N72tdnRkXePRRra+4QuuJE+v7DloHoH9BE30gHNgN9AOigJ+AoR7a3wy8Zn9/NrAYiAA6A2uArp6OJ6IfRFauNANU0dFav/66b9tMmaL1iSc27XjPPGMuqb17Pbd77DHTbuPGph3HE0VFWnfpYgT+wgsb/91yi/dB7B07tI6IMIOdbQ3LOj3lFP+3tYT3L39x32bePNPmnXf827fNZtyFI0a4vmEUFuqguI9c8cQT7m82jnzxhdY9ehg3k+Xic37i8UR5uXFd3Xmn/32sqDA3xSFD6gW+Tx/jDr3hBvN///xzrTMz/d+3nWCK/iTgK4fP9wL3emi/Cphpf38XcL/DuleBizwdT0Q/iEyapHXfvv6J61NPmcuiKY/4c+aYC9mblVhYaG5E11zj/zG88cADpv8bNgS2n5tvNlbgli3B6VewuPJKc35JSf5tZ7NpffLJZjtPT3u1tcba7NXL9dOAO777zvTrn/903yYxsXm+83vuMWLsy9PJvn1aT55s+jp0qNaVlf4d69RT/bPGCwuNVd+zpznmqFFa/+tfWpeW+ndcHwim6F8IvOLw+XLgWTdtU4H9QLj98xnASqATEA9kAne42O46YC2wNiUlJej/jHZLr17+/8h27jSXxT/+4f/xUlK0vugi39pef73WHTpoffCg/8dxR0mJ8Smff37g+yoo0LprV61//vPA9xUsPvvMfDeWgPgjWJ9+arZ54QXvbTMyjJvv7rt93//tt5vv05O/e+rU5hkIvfZa8z/xlepqM461Y4f/x/rjH83/xtsNsbbWPE106mT+77Nmaf311806kO2r6Ac7ZHMO8L7Wug5Aa70I+Nxu/b8NrAYaFbXQWr+ktR6ntR6XkJAQ5C61U6qq4MABSEnxb7sBA0x43cKF/m2Xk2OSfByTsjxx662mjy+84N9xPPHMM3D4sEmwCZT4eLj3XpNotHx54PsLlMOHTRjksGEm7BV8L2tgs8F995nv9uqrvbcfM8ZkpH7xhe/9W77clC/o0sV9m2HDTIKWDnLW86FDJmvWVyIjTcLVwIH+H2vaNPP//PZbz+0++QT++leYPRs2bjT/y9NPB6X8P2aQ8UX0c4G+Dp+T7ctcMQcj7sfQWj+mtR6ltZ4JKMxAr9Dc5OSYV39FH0zphOXLTXatr6xaZV4nT/at/eDBpu7L888Hp+Z+aSnMmwfnnAOjRwe+P4BbbjHx6//4R3D2Fwh33AH795tYckussrN923bxYiM8Dz3kPg7dmalTzTaFhd7bHjli6heddprndkOHmrZ5eb71wVcOHTIZ3y3BSSdBx47e4/Vfegn69oW33oLhw1umbz7ii+ivAQYqpdKVUlEYYW9kBiqlBgNxGGveWhaulOphfz8SGAnIFDotwd695rUpoj97tknS+fxz37dZuRI6dYITT/R9m9tug4MH4e23vbf1xvPPmx9/MKx8i+houPxyY+0XFARvv/7y1Vfw6qtw110wfjykpprl1nfsDav8wRln+H7MqVPNqzeLFsx3b7PBqad6btdcNXhaUvQ7dDBPs56K1GVlmZnCfv3rhglqbQSvoq+1rgVuAr4CtgLvaq03K6UeVkqd69B0DrDA7luyiARWKKW2AC8Bl9n3JzQ3lhVoCYQ/TJhgauF/8onv26xaZbbz1ZIEmDHDPPLPmxfYI395uXmU/tnPGmcCB8rcueYG+O9/B3e/vnLkiMmUHTwYHnzQLEtONm4CXy39rCyTge2PC2TcOHMTX7bMe9vly833PmmS53bDhpnXYNfgaUnRB+Pi2bDB/VPQyy+b78cXV1or4JNPX2v9udZ6kNa6v9b6MfuyB7TWCx3aPKi1vsdpu0qt9VD730la6/XB7b7gFksQkpP93zYsDH7+c+OH9MX1Ul5uHu999edbKGV8+xs2+CYu7njxRWOJB9PKtxg2zFjXrVWa4e67TemM+fONWwFMyYikJN8t/awsSE/3z58cFWVcdb6K/vjx5ibhiYQEM1biTfRvv924R3ylqKhlRX/6dPPqaqynpgZee824GZvy22sBpPZOqJKdbaz1Dh2atv3s2cZP7ssg5po1ZtIJX/35jlx6qRGCefP83xagogKefNL8EJtyfF+46ipTFnrduubZvzuWLDE3tNtuM75kR1JS/LP0+/Xz//i++PXLy2HtWu+uHQtvNXiys+HvfzdVL32hshKOHvXvKSZQxo+Hzp1d+/UXLjQuy+uua7n++ImIfqiyd2/T/PkW06cby80XF481iOssTL4QHW0iKT77zBQC85dXXjFRSs1h5Vtccom5ec6f33zHcEZrMwHJoEHwyCON16em+mbpa11v6fuLL3791auN+8vbIK6Ftwief/3LrPNlABmguNi8tqSlHxlpCrC5Ev0XXzS/u1mzWq4/fiKiH6pkZzfNn28RHW185AsXeve3r1wJQ4Y0/Yf329+aH5K/UTJVVWYikClTfBedphAXB+edZ/z6wYg08oUdO2DXLhO1Ex3deH1KignZ9DYFYX6+sYSbIvq++PWXLzfuQF+fsoYONVFhBw40Xqc1vPmmee+r6B86ZF5bUvTB+PW3bDFWvUVmpomUaqMDuBYi+qGI1kb0A7H0wbh4cnI8T2xusxlrLxDXSq9expqeP7/ecvOF+fONv/v++5s//nnuXCMwn37avMexsIR22jTX61NTzQ0oP9/zfrKyzGtTRN/y63ty8S1fbuL6PcXnO+IpgmfNGti+3fj+jwfRh4Y3xJdfNmLfRgdwLUT0Q5HCQuPrDFT0zz7bWHGeErW2bzdC7e8grjO33WYsUl+TtWpqTP37iRNNFFBzM2MG9OnTcgO6y5aZeVoHDHC93vpuvfn1AxF9MC6eDRvMYKkzlZVmEnF/nrI8RfC8+aZxo1nTN1ZWet+f1a+WFn3rRmeFblZX1w/g9unTsn3xExH9UCSQGH1H4uON79KTX3/lSvMa6CDqiSfCzJlmCj9fXCgLFpjz/P3vWybLMTwcrrjCRDTt39+8x9Iali1DnzbV/bn5GqsfDNEH13797783YueP6CcmGoF2Fv3qapOvcd55Zl5dcH2jccay9FtyIBfMhCqnnVbv11+40Dx1XX89ALV1Nj7bkMfR6rYXoS6iH4oEEqPvzLnnmsiVPXtcr1+1ytwcmpLS7sxddxlfr7eYeJvNWPnDh5unkZbiqqvMsd96q3mPs2MHHDjA32qS+Oui7a7b+GPpJyaaaJOm4Mmvv3y5uSmdcorv+1PKdQTP558bAb/iCnM9gW8untZy74Bx8ezcaVyM1gDuGWdQVlXLNW+s5ab/rOOvi9peAQIRfX/Yvt1YtL6GyrUWVv8CtfTB+PWhsYunthbeecdE3Zx8cnCs7RkzYNQoeOopzwOUn35qLMV77zXupyCgfUkOGzTIfP/z5we/foxjX+zW46dxg3hu6S425Lgoh9GtG3Tt6t3Sz8xsupUPnuP1ly+HkSPNQLc/DBtmRN/xf/jmm+bmdMYZ/ot+RIRJPmtpLL/+K6/A11/DtddysLyGi15Yzf92FTI0qStvfbeXA4d9cFO1ICL6vqK1iTJZtQp++KG1e+OZ7GxjnQXD+nEuwFZaaqJsBgyAOXOM+Nx7b+DHAXPjuPNO2LrVfQkIrc2crv36mTl3g0DG3mJGPriIjL2HvDeeO9fMA/v990E5tiv2ffQFB2K6c84FpxIf04F7PthIbZ2Lm6AvsfpNDdd0xJVfv7raDOA3JWpq6FAzDmQNQhcVGePh0kuNgPsr+t27t04hsxNPNDe8xx6D8HB2//wizn9uJXuLynn1ynG8ePlYbDbNs0ubEIrcjIjo+8q779YP2gS7YFSwsWL0g/VDmD3bWHV33WX2e+utppjUxx8bAWxKfL47LrrI7PvJJ12vX7bMCO5ddxmBCJCaOhu//2gjpVW1vLA80/sGv/ylCaFspgHd7MJyoletYNeQcdx2xgk8dO4wtuw/wvyVexo39harX1trbgpNScxyxBJ2R7/+2rUmMa6pog/1Lp533jED81dcYT77I/pFRUHx51dU17FiZwFPfrWNT9a7qyfpRFiYOf+aGoqm/4zzPsqi1qZ55/pJTD0hkb7dO3HR+L68s2Yf+w4dDbiPwUJE3xdKS01q+OjRJp68uQfyAiXQGH1nrAJsf/ubccGsXg0rVpjlQXKvHCMy0txUvv3W9RPV44+bEM+rrgrK4eavzGLbgVLGpcbx9daD7C0q97xB165w4YVmILmiIih9sKits/HkswtJKCtm2KXnEh6mmDW8FzOGJPK3xTsaC4c3Sz8nx2RKB2rpjx9vbnSOLh4rjHPKFP/35xzB8+abxk00apT5bD2h+mPp+0l1rY01ew7x9693cNGLqxn50Fdc/uoPPLd0N7csWM+DCze7frpyxl6S4e7uE0mK7chHN05meJ/YY6tvnj4ApRTPfNN2rH0RfV94+GFj3T//vBGctm7pByNG35EJE+CDD8wA43vvBdeyd8W110JsbGNrPyPDJL/cdlt9HZoAyC2pYN7incwYkshzl44hXCneWOVDluvcuaa+va+lAnzkn8t20+U7Ew0Vd5apiKmU4qHZw1EK7v9kU8Oxh9RUI3plZa53GGjkjoUrv/7y5cZib8r8F716Gbfgli1mnOz77+utfDBPcHFxAYt+bZ2NDzJy+Oui7fzu/Q3Mnf8DZz+9gvGPfc3g+7/gly+s5h9LdlJRXcfVk9N54+oJbHzwDK6dks7rq/Zw9RtrOVJZ4/bQBaVV3Nt9Ar+ZfQ9Hp83kvRtOpk+3hol0SbHRXDoxhQ9+zCWzwM331MKI6Htj82ZTC+Saa4zY9e7tn6VfVGQqJQaDPXu8DyBWVBhfaTBFXym44IL6ULrmpksXU4Lgww9h9+765Y8/bsTihhuCcpgHFxr3woPnDqNn146cPTKJd9fuo9TDDx0wj/T9+pnwvNtv973wmQd+2lfC35fs5BdHdjWKz+/TLZo7zziBZdsL+GyDw7Vnfcfujp9pd1cFKvrQ0K9fW2tCdX1w7VTW1PHlpv0NBc8xgufNN83T4q9+1XDD+PgGom+zaZ5ZspM73/uJyhqHeZjciL7Wmvs+2sgd7/3Ec0t3sXR7PvmlVSR26cD0ExK5cdoAXrhsLOvun8mnN5/CvWcN4bRBCXTpGMnvzx7Kny8Ywapdhfzi+VWNnrCqa228/G0m059axvtbiki57greuGYisdGuK8z+Zmp/osLD+McSz9b+2z9k88Ly3R7bBIPAnaKhjNZw001GhB5/3CxLSvKvRszs2ZCWZmqKBMLu3SYs8tNPPYcpWrMpBVP0W4P/+z/jTpo3D5591owdfPihmQGqa9eAd79o8wEWbznIPWcOJjnOVIecOzmdT9bn8X5GDnMnexDKsDAz0PzIIyav4Omn4eKLTcmEMWP87svR6lpue2c9iTFRjM78CU6f3mg85sqT0/h4fS4PfbqZUwcmENspst6Fl51d7zJxJCvL9LVv38brvJBfWklEWBjdO0eZBY7x+snJ5unCi+hnFx3lN//OYHOeMXrSenRi6gmJTB+cyOQhQwn/+CNjyJxxhvldOeIg+hXVddzx3no+33jA3rcqXrp8LB0jw92K/lOLtvPu2hxunj6AW2cMIjzMv/GtORNSSOnRid/860dmP7eSFy8fy/i07izdns8jn20hs6CcaSckcP85Q+mX4DlyKLFLR644OZWXvs3kxmkDGNSzYfZynU3z+OdbeeV/WUw9IYFrp/Tzu7/+IJa+JxYsMI+0f/pT/WOsv5b+1q2mUmGgZGaam5CVDOWOYMbotya9e8Nll5ksx8JCeOIJ49K55ZaAd11eVcuDCzdzQs8uXHNKvbiP6tuNMSndeGPVHmw2L09UJ5xgbuSZmWYM4tNPYexYMyWet+/Iicf+u5WsonKem9CFsIMH6gXWgfAwxZ/OH0Hx0Rr+/OVWs9CbpZ+VRXWfZG5+fxOjHl7ELQvWsWp3odtzs9k03+4o4NdvrGHin5Yw5pHFTH9qGXe+9xPv0BNbdDR62bJ6f76HyppLt+VzzjMr2HfoKH+/eBQPzx5GWnxn3v4hmyte+4En9oWZp4bsbPTllzfegV30Dx6p5OKXVvPFpgP84ewh/OUXI/h2RwHXv5VBZXmFufk4DeTOX5nFc0t3c8mEFG6f6b/gW5zcP56PfnsysdGRXPry91z84mrmzl+D1jD/qvHMnzvBq+Bb3HBqfzpHRTBvccO4/bKqWq57cy2v/C+Lq05O45UrxjWr4INY+u45csRYbmPHGh+zRVKSsS4qK737lY8eNW1raoxgBxJNY4W3eSvvG8wY/dbmjjtMTPx995mEqN/8pmk+ZCeeXrKTvMOVvH/JaCLDG9o9cyenc/Pb61i6PZ/Th/T0vrOUFJNXcP/9pvbKvHlw1lmmvn9UlNfNP1qXw7+/z+baKemMyfyfWehC9AGG94nl6slpvLwii1nDk5iS3ouwiIhGg7k2m2b5jgL6fLeBQtWNpdvymTIwnm+25fPJ+jxSunfi4vF9uXBsMj27duRIZQ0fZOTw1uq9ZBaWEx8TxW+n9iemQyQZe4v5Zls+72dU0zvxBBLe+YyqpN70T+lHRHwizr+AOpvmH0t28vSSnQxN6soLl40lpYd5krpiUhoV1XWsziwku2MuLILSqGiuzEvguk37OWNoL8IswYuPp2ZNBrOfXcmRyhpevnwcM4aa70NruOfDjfzu5WX8AxpY+gt/yuOhT7fws2E9efS84agAI9j6JcTw0W9P5rf//pENOYe576zBXHVyOlER/tnLcZ2juPqUdJ5espNNuYcZ3ieWnOKj/PqNtezML+OR2cO4fFJaQH31FRF9dzz0kMkO/fjjhhXzevc2rwcOGLeNJ3LtoV+lpSYuOZC4eUv0f/zR8w0kO9usa+P1P3xi2DAjoC+/bAb37rwz4F1uO3CEV/6XxZzxfRmX1vj7mDW8F726duS1lVm+ib5FbKzpX//+Zvzjhx+8Zqp+n1nE797fyEn9unPXzwbDFY8ao8JdvR3gtpmD+GLTAa587QeiI8NZ1jWBnG/X893SXQxIjKHkaDWvrMhiZ34Zawvy0FNOZ9W90+naMZKK6jq+3Lyfd9bs48mvtvPXRdsZl9adTbmHOVpdx+iUbvz94lGcOaIXHSLqr3mtNXuKjlJ+ZCaD//kk5cX7WTh4Cn969GtmDe/FeaP7cFK/HhypqOGWd9bz7Y4CLhybzKPnDTcuGAeio8KZPrgnXHs2PASFs86lmEhu+NePDEyM4abpAzhnZG+yiaZ3QQFhaN6/4WSG9q536c2ZkIIGXn3pvwDUxHYzU/TtLOCOd9czIb07/5gzOmgWc7dOUfz71xOpqrU1Oh9/uOaUdN5YtYe/Ld7BTdMHcN2ba6mqtfH63PFMGRi4MeMrIvqu2LzZJCD9+teNp9+zfI95ed5F35qcHIx/NRiin59v3EvWzceZvXtNH6OiyCwoo86mGdjTxwqIbZG77jL+80svbfT0UlBaRdfoiAYC5QmbTfP7jzYRGx3J72YNdtkmMjyMK05O5Ykvt7P9QCkn9PLzfzd1qvGjf/21R9HPKizn+n9lkNw9mhcuG0tUuDKuxKlTPT4RdoqK4IPfnMySrfnsyi/j0Lu9CM8xIm4xJKkrz5x7AvF/OUT8pJHQ0QwwRkeFc/7oZM4fncyewnLeXbuPxVsOcvaIJK6YlMaI5FiXx1RKkR7fGS6dDf98ks7VFYy5bDY/S+vFF5sO8F5GDj27diBMKYrKqvnT+SO4ZEJfz1Z2nz7w8sukz5rF17378NmGPJ5buotbFqzniS+38/Oso9xTV8Mnc0eRkNR4DOeSCSnEr0uEV+HvPxYy7eRDXP9WBv0TYnj5inEBibO7/0Gg+4yNjuS6U/vx5Ffb+d/OQnrFdmTBdeMYkNiyv08RfVd8/LGJb37sscbrLLH1xa/vKPp79hhXUVNxrNu9bp170bfH6JdV1XLxS99RXF7NbTMHccNp/ZvdV+gLm/MOs3jLQYYkdWVMShwJXbzM7HXaafCvf1Ez/XS25RwmY+8hMrJL+HFvMbklFfSO7chjF4xg2gmJHndTcrSaR/+7lYy9xTz1yxOJ6+ze9XLJ+BSeXrKT11dl8fgFI1222WIfnHS0QAETajhunBF9a05bJ4rLq5k7/wfClGL+VePp1inKhMPu3+/WteNIz64d+dVE+w3wveGwbBkbHzyD3QXl2LRmdN9uqK12v7+byJ20+M7cPWswd7u5+bnEitevqOCEi8/hqZQUHj1vOEu25vPRulwKyqp44bKxnNi3m2/7+/WvAQgHZo/qw89H9mbRloO8+O1u0ganwjJIqCoD4l1uPjPJ3MyWF9p47oXVJMdF8+bVE9xG0bQFrjo5jX9/t5fk7p144bKx9QPlLYiIvityc83gkCv/saOl7w1nSz8Q8vON62D3biP67iJ4srNh7Fie/WYXBaVVnDoogSe/2s7yHQXMu3hUozhiR2w2TWF5FYldAo+Bd8XnG/dz+7vrqaypT3pJ7dGJsSlxjEmNY1jvrpRV1ZJ/pIr80irySyspKK3iwJF0Nj+/ngp7qF5SbEfGpsZx+aRUPsjIYe78NZw/ug/3nzO00Y9Ia80HP+byp8+3criihhtO688vxnh2fcV1juL80X348Mdc7vrZ4Ab73H+4gj9/sY1P1pvv/6R+3bnu1H5MHZRY748+/XSTY1Ba2qjOfFVtHde9tZa8w5X859cTSe1hL4RmxcD7IPoNSE2F3Fy6RChGOYqtdb0Fmo3rSFSUGbzdtevYU1fHyHDOHpnE2SOTvGxmzrDZAAAgAElEQVTsnTB7Mtqs4b3g00PwAmYQ390Ttb0sxHXnjePFbBvPXDKaxK7Nc+0Gi84dIvjmzql0iAgLeLyhqYjouyI3171PPD7e+Jd9tfS7dzfFw4Ih+gMHmkd/d4O5Nhvs20fJrHN49X+Z/GJMMk/9ciQf/pjLA59sYtbfv+VP54/g5yc2fErYduAIH6/LY+H6XPIOVzK8T1cuHp/CuSf2DorVpLXmuaW7eGrRDsamxvHMJaPZf7iCjL3FZOwt5tudhXy4rnHqe+eocBK7diShSwcuHt+XsalxjE2No7fDjWvu5DSeW7qb55fu4tsdBTx47jDOGZmEUoqdB0v5/ceb+CHrEGNSuvHY+SMY4sJV4IqrTk7n7R/28fYP2dw4bQCVNXW8siKT55bupk5rbp4+gK4dI3ltZRZXv76WAYkxXDslnfNG96HDjBkmxPfbbxvcnLXW/O79DazZU8zTl4xuOKawbJkxKPytVpqSYr733NyGEVvBSsxy5pVXTIBCc+NLKQZ7hc1zp4/g3CCE8bYUwXY9+YuIvivy8ty7T8LCfM/KzckxMc3h4e5LE/uAzaY5ui+PFXXdGN1/CL3ciX5+PlRV8VlJJFFxYfxu1gkopfjF2GTGpcVx6zvrj0Wm/HZqf77ems/H63LZdqCU8DDFqQPjuWRCCp9vOsD9H2/i0c+2cNaIJC4e35eJ6d2bZJlU1dZxzwcb+WhdLueN6s2ffzGSjpHh9O4WzdhUI3paa/YdqmD7wVJioyNJ7NKBhC4d6NzB++XZISKc22cO4szhvfjdBxu4+e11fLI+l/6JMby6IovOHSL48wUjuGhc33pL3AdO6NWFUwbE89bqvaTHd+bxL7ay71AFPxvWkz+cPZS+3U1EylWT0/jvhv289G0mv/tgI09+tYMLhvbg7qgOZC/4mMKhk0js0oHErh14cXkmH6/P484zBnGu443XXj/fmz/fJY6x+s6iHx0NPf0YjPaF5OTg7s8dvop+eLjvs3YJgIi+a/LyTC0QdyQl+W7pJyeb2YC2bWtSVzILyrjn/Q28UVjIgfSu/Kc6htuzssw8o92cfKf20L2lFR35v9MHNnjUTe3RmXevn8QzS3by7NJdfPijsaxHp3Tj4dnDOHtEEj1ijH/9pukD2JR7hAVrslm4Po+P1uWS0r0TAxNjiI2OpKv9L9b+16dbNAMSY4iPiWpwYygsq+L6tzLI2FvMHTMHcZO9DokzSilSenQ6FtrXFIYkdeXD35zM/JV7+Ovi7Xy9NZ8LxyZz75mDj52Xv8ydnMY1b6zlt//+kUE9Y/j3rycyeUBD/3JkeBjnje7D7FG9WbW7iJe+zWR+xn5O6T2UxC8Xc1Gf8xq0/8WYZG6c5hSds3Onz/78RjjG6jvWwcnKMm6RVnIhBIyvot9aFTaPY0T0namtNeGYnkIee/euT3H3RE6OGdSLiYEvv/QrVr+2zsbLK7KY9/UO4mxVRNdW8cuzxvLHTLN9wYrvSPj5rAbb1GTtIRJQKSkuM0ojw8O4/YwTmDo4kXXZJcwYkljvU3ZAKcWI5FhGJI/gD2cP5YtN+/n0pzz2H65k+8FSDlfUUFrZeEag2OhIBibGMCAxhvT4zrz13V4KSqt49lejOWekmyenIBIRHsa1p/bjzBG9KC6vcRuN4ivTTkjk8pNS6Z/QmctOSiUi3H1stlKKyQPimTwgHptNU9lxPZ0e+D1v/zyV3I7dyC+tJCo8jCsmpTW+8TXVnw/uJ1PJzAyuP7+liY01Vrwvoi/4hYi+M/n5xkfqzr0DxtL/3/8878eatDo52VjkVk0cHx63N+cd5ncfbGBT7hFmDevFoyM7weMQ07c3N513Csy/lw9e/YxfzTydrh3rfe4ZK37iJOCKi6d4TB4ZkxLHmBTfJr6IjgrngjHJXDCm4WN9nU1TVllLSUU12YeOsiu/jF35ZezML2PxloMUlVeT0KUD71w/qeEAYwuQHNeJZD/n9XBFWJjikfOGN2m7Tmf9DB74PZP2bmhcV8aZpvrzwcybEB/fMCtXa2PpN6UCZlshLMwEU3gS/SCVVW5viOg7YyVUebP0i4qMsHdw4zqwfP7JyfVRQHv2eBX9DzJyuPuDDcR1iuKfl47hzBFJppQxQGIi6cP7U53Yk567t3Ljv3/ktavGExkeRkFpFbvWbObEjp04dcIg38+3iYSHKWI7RRLbKZLUHp0bJZccKq+mU1R4qw9atRqjRhkr9OuvPYt+IP58i9TUhpZ+cbHJKA/2IG5L41R0rRGHDoVGEmILI7V3nLHE2pulD8YN5A4rXDM5uT7kzEsEz5KtB7n7gw1MTO/O17efagQf6hOzEk0setS4sUyryGXFzkL+uHAzWmue/GobiSX5hKeltgkfZ/fOUe1X8MG4JqZNM6LvqTLqunVN9+dbpKQ0tPSbK3KnpfFF9MW94zci+s74aumD58FcP0U/Y+8hbvzPjwxN6spLV4wzCTsWluhbTwmjR9Ntzy7+b1If/vN9Nvd8sJH3MnIYUXeYqPQ0930SWpYZM0zV01273Ld55BHjv/7lL5t+HMvSt24uIvqCB0T0ncnLM1aap8JeviRoOYp+TIzZn5uwzR0HS7n69bX06tqR+XPHE+McqmiJvtWnMWOgro5be1Vz9sgk3lm7jx6dO9DzcJDr6AuBMWOGef36a9frMzJM9vftt/s/ubgjqalQXn4sbj2odfRbE0+iX1Njkt/Ep+83IvrO5OYaUQ/34Jrw1dLv2rU+hjgtzaWln1dSwZWv/UBURBhvXTOReFfhhfn5xhq0xg9GjwYgbP06/vrLE7loXDJ/Pas/YYWFx39J5VCif3/zfbgT/T/+0Yj9rbcGdhznEstWnafYwKKXWh1L9F25x6wbnFj6ftM+RP/WW01ddl/wlJhlkZBgbgreLH3HRJb09EaWfnF5NZe/+j1llbW8MXfCsYSfRuTnH/PnA+YG0q0brFtHx8hwnrjwRE7rVGXWiaXfdlDKlGT45htTy8mR77+H//7XFJQLNJvUMUELjOgf71Y+GNGvqzNTUzojot9k2ofo/+c/8O67vrX1VILBwsrK9WbpO4p+WpqxxGym7szR6lrmvr6GfcUVvHzluMaFuxxxFn2lTHSIY2ZuKNXRDyVmzDCJdD/+2HD5H/9oRO2mmwI/hitLP1REH1y7eET0m0zoi77W5kfna+0bXyx9MC4gfy396upj2/zli21syCnh6TmjOKmfF7/kwYMNRR+Mi2fDBpNMBiL6bZXp083rkiX1y1auhK++grvvDk4Jgfh4U3IhO9sYFXv2HN+JWRYi+s1C6Iv+0aNm0GfPnmNWtlsqKkyMsy+xv56mTaypMeucRR9gzx4qquv48MdcZo/qw6zhPlQndLb0wYh+ZSVst9dR37vXPIFI3HLbomdPGDGioV//gQfM8htvDM4xlKoP28zLM8ZFe7H0ZSDXb0Jf9EtKzKuDle0WX2L0LTxZ+gcOmCcMZ/cOQFYWX27eT2lVLReN82HC6ro6c9E7J3XZB3OPuXiys43gR0i+XZtjxgyTwV1RYRKxvvkG7rnHZNMGCytsM1TCNcGz6NvLKoul7z+hL/rFxfXvvdXL8SVG36J3b3MxVlc3XucYrmlhDbbt2cN7a3NI6d6Jiek+XLBFReYG4mzpDx5s5uh1FH1x7bRNZsww2dsrVxpffu/ecP31wT2GZemHougXFDReZ1XYPI5KKrcVQl/0LUsfvPv1/bX0wXVWrivR79gRkpIo27aTVbuLuHBssm+lfp2ycY8REWHcBiL6bZ9TTzXf1wMPmBr7991nfPDBJDXVXCtbtxp3TyiE7sbEmIlb3Ll34uLaRPb58YZPoq+UmqWU2q6U2qWUusfF+nlKqfX2vx1KqRKHdU8opTYrpbYqpZ5WLT1djKPoB9vSB9d+fVeiD5CezqHNO1AKfjHWx7rk7kQfTJLWunXGBbRvX2j80EORmBg46SRTQyk5+dg0gUHFuuEvX26uX3c1oY4nlHKfoCXZuE3Gq+grpcKB54AzgaHAJUqpoY5ttNa3aa1Haa1HAc8AH9q3PRmYDIwEhgPjgdOCegbesNw7YWHeRT8vDzp1YlVBNZe98j2FZVXu23rKys3JMf5ap3r3Oi2diOy9nDIg3uO0hQ3wJPqjR5ub2vffm8FjsfTbLlZ27h/+0DyCbN3w16wJDdeOhSfRl0HcJuGLpT8B2KW1ztRaVwMLgNke2l8CvG1/r4GOQBTQAYgEDrrZrnmwLP2hQ31z7/TuzSfr9/O/XYVc++ZaKmvqXLd1sPS11ny1+UD9TcIK13R6qMmJTSSxJJ+LRvkxn6g1Ibo70Qf45BPzKqLfdrnuOnjoIZg7t3n2b333tbXtQ/SLisTSbyK+iH4fYJ/D5xz7skYopVKBdOAbAK31amApsN/+95XWequL7a5TSq1VSq0tcDVoEwiWpT96tG/unT59WLP3EH26RbN+Xwl3vPcTNpuLNPCEBAgLQ+fl8ceFm7n+rQyufXMttXW2xjH6dv5X14UIbeOMrjW+9z8/3wxYuarNMmKEWSei3/ZJSjI+/ago722bgqOR0R5EX9w7TSbYA7lzgPe11nUASqkBwBAgGXOjmK6UajSzg9b6Ja31OK31uARPhc6aQkmJ8akOHGj87xUV7tvm5VGV2JPMgnIun5TKPbMG898N+3lq0fbGbcPD0T178sPKTby5ei+nDkpgXXYJTy/Z6VL0D1fU8GWZmb6wQ0524/25Iz//2A2mEdHRJorHitUXn377JTKy/ulTRF/wgC+inws4BpQn25e5Yg71rh2A84HvtNZlWusy4AtgUlM62mSKi41v3cpQdDdBudaQm0teZ+MnHJ8Wx3Wn9uOSCSk8v2w3767Z16B5ZU0dezt042h2DneeMYg35o7nwrHJPL9kO7a8vEai/+lPeWR2SfTcB1d4m23LcvHExkr4WnvHuumHQjauRXy8+Q1bmedgxq+OHBGffhPxRfTXAAOVUulKqSiMsC90bqSUGgzEAasdFmcDpymlIpRSkZhB3EbunWalpMS4Rqwfgju/fkkJVFayK6ILURFhDO8Ti1KKh2cPY8rAeO77aCMrdxmLo7yqlmveWMOu8BhODDvKTdMHopTiwXOHMTKqmrDaWioSG/rt31u7j64D09FhYb6XhADX2biOWKIvrh3BugZCzdLXumG+jTVOJ5Z+k/Aq+lrrWuAm4CuMYL+rtd6slHpYKXWuQ9M5wAKtG9RBfR/YDWwEfgJ+0lp/GrTe+4Jl6Vs/BHd+fXu45npbZ05MjqVDhCmtHBkexnOXjqFfQmdu+FcGGXsPcdmr37N6dxH9Rg6i+5GiY7uI6RDBnyeaC/H1fbVY/4rtB0r5KecwF0xMRyUni+gLzcPw4UYIfckzOV5wlZUr2bgB4ZNPX2v9udZ6kNa6v9b6MfuyB7TWCx3aPKi1vsdpuzqt9fVa6yFa66Fa69uD230fsCz9nj2ND9yd4NpDLzNqohmX1vBi6toxkteuGk+HiHB+8c/VbM49wvOXjqXfyAFGlGvqB2YH1ZgysJ8VhfFehonXf2/tPiLDFeeN6u2yxLJHvIn+qFHmVURfuOMO2LTJ9fjP8Yor0ZdiawERQleHG0pKjKWvlBFcL5Z+TqfujE9rHCmTHNeJV68cx6i+3Xj1qnHMGt6r3qI66BCFak/M6jN8IA8u3MyOg6V8tC6XGUN60iOmg9vJVFxy9CiUlXkW/bg4mDfPhAQK7Rt71ndIYQV2iOgHjdAX/eLi+nDHfv28WvoFMd0Zm+L6Yjqxbzc+vnEyUwbaL0RXCVo5ORAVxUPXnEZURBgXv7iaovJqfjnOPrCbnm7aV3lI/LLwlJjlyK231lv8ghBKeLL0ZSC3SYS26NfVmVF+KzPWsvRdTb+Wm0tpTCxpfXoQ2ynSt/27KsVgD9dM6taJP18wkuKjNSR26cCp1o0iLc0cP9uHsE1fRV8QQhVL2MXSDxqhXYfXmmbN0dIvLXWZwq1z89jfKY5xLlw7bnFn6dvDNWcN78XDs4fRs2tHIsLt91drQDkry+QOeMISfU8hm4IQykRHQ+fOjQdyw8IkRLmJhLboW6FdjpY+GGvfSfQr92aT17kH49P8sB4SE83F52zpn3TSsY9XTEpruI3DZCpeEUtfEBonaFkVNkNpwLoFCe3/mhXba4m+h1h9nZvHwZju/ln6ERFGkC1LX2u3JRiO0bu3yZ70ZTDXEv1gZykLwvGEK9EX106TCW3Rtyx9y73jLla/tpaORQWUxff0vfqlheO0idakKp5EPzzchFf6YukfPGhKSARzhiVBON5wJfoyiNtkQlv0nS39mBhjNTuLfn4+YdpG57S++F3u33HaRHd19J3xNWzTW4y+ILQHxNIPKqEt+s6WPhhr30lw87ftBiBhcBNqljha+r6Kvq8JWiL6gmBE37H6rpRVDojQFn1nSx+MX9/J0s/6aScAaSO9RNO4IinJiHNtrX+if/CgSb7yhIi+IBjRLy2tz20RSz8gQlv0S0qMDz0mpn5ZerqJkXeo2ndwu7kJpI1oguj37m0GcA8eNKJvDe56Ii3NvHqz9r1V2BSE9oCVoFVUZH63hw+L6AdA6Iu+VYLBol+/hlY5UJ6ZTV1YOOG9miCwVqz+/v1mn717mxuNJ3wJ27TZzCOtWPpCe8cxK9dy2cpAbpMJbdF3LMFg4ZgcBRw+WkP4gf1UdI/3LtausLJy8/K8h2taWJa+p8Hc4mKTUSyiL7R3HEVfsnEDJrRF37L0HbFi9e1+/R+zi+lVWoTu43IGSO84W/q+iH6vXqY4lidL39PcuILQnnAUfSmrHDChLfquLP2+fY1Fb7ey1+49RM/yQ3RK7etiBz7Qs6dxH/lj6StlZjnyZOlLNq4gGMTSDyqhLfquLP2ICJMcZbf01+wppnf5IcL7+iDWrrAGbrdsMdE4vog+uAwdbYCIviAYHIuuSYXNgAlt0bdmzXLGXmK5qraObZkH6XK0NLDZhpKS4IcfzHtfRX/QINi2zWTwukJEXxAMkZFmDmix9INCaIu+NWuWM/YSy5tyj9CtxJ7p11SfPpgbhlUq2VfRnzbNPBl8953r9fn5pqCUWDSCUJ+VW1Rk3KOxsa3do+OW0BX9igqTzOHO0s/PZ/3WHHqW2QeGArX0LfwR/fBwWLzY9fr8fHOhNyWiSBBCDUv0pcJmwITuf85VCQYLe9hm3rotDKfMLAvU0gdzIfbq5ds2sbEwYYJn0RfXjiAYHEVfXDsBEfqi787SB2p2ZzLYZhf9YFj6vXoZ/6OvzJwJa9bUl4tw5OBBEX1BsHAUfXF5BkToir4lpB4s/ajsPaRVFZvSxYH4CK0bhq+uHYuZM03m7dKljdeJpS8I9YilHzRCV/Q9Wfrx8dg6x9D70AF6lRUb0fa3pLIjlqXvr+hPnAhdurh28YjoC0I98fFmnG7fPhH9AAnd6RI9WfpKcbRPX/oePkBcWFhg/nxouqUfGQlTp8KiRQ2XV1aaCd1F9AXBYCVoHTggoh8g7dPSBw71TCal5ACdC/MD8+eD8eWnpxvL3V9mzjSJYo7lnq3a4VJhUxAMluiDiH6AhK7ou6ql70But570PXyQsAP7Axf9iAgj2r/6lf/bnnGGeXV08UhiliA0xFH0ZSA3IEJX9EtKzABtVJTL1TtjEulUU4WqqAjcvRMIgwaZekAi+oLgnoSE+vdi6QdE6Iq+uxIMdjZGOlw4gVr6gaCUcfF8840ppQxSYVMQnBH3TtAIXdF3V4IBqKypY32Uw4XTmpY+GNEvLoaMDPNZLH1BaEi3bvVZuCL6ARG6ou/B0t9TVM6+rg6C2pqWPsDpp5tXy8WTnw/R0dC5c+v1SRDaEuHh9WIvPv2ACF3R92DpZxWUUxnZkZpEe3RMa4t+QgKMHt1Q9BMTA8sdEIRQw3LxiKUfEKEt+m4s/czCcgDC+vUzF1DHji3ZM9fMnAmrVkFZmUyILgiuiI+XCptBIHRF39WsWXYyC8rp2bUD4dOnwSmntHDH3DBzJtTUwLffSjauILgiPt4YclJ5NiBCMyPXZoPDh91a+lmFZfSLj4H7HmvhjnnglFPME8eiRUb0R49u7R4JQtvitNP8K2gouCQ0Lf0jR0Br95Z+YTnpCW1skLRjRzj11HrRF0tfEBpy663w7rut3YvjntAUfQ8lGIrLqyk5WkO/+DYm+mBcPFu3GjePiL4gCM1AaIq+h2Jr1iBuv7Zm6YMRfQsRfUEQmgGfRF8pNUsptV0ptUspdY+L9fOUUuvtfzuUUiX25dMclq9XSlUqpc4L9kk0woOln1lgJk1Jj49p9m74zYgR9WIvoi8IQjPgVfSVUuHAc8CZwFDgEqXUUMc2WuvbtNajtNajgGeAD+3Llzosnw4cBZzqCDcDHoqtZRWWExGm6BsX3ezd8JuwMJgxw7yXkE1BEJoBXyz9CcAurXWm1roaWADM9tD+EuBtF8svBL7QWh/1v5t+4mF+3MyCclJ6dCIivI16ti6+2MQhp6a2dk8EQQhBfFG+PsA+h8859mWNUEqlAunANy5Wz8H1zQCl1HVKqbVKqbUFVi35QPDg3skqLG+bg7gW555rpoSTBBRBEJqBYJu7c4D3tdZ1jguVUknACOArVxtprV/SWo/TWo9LcCyh2lSKi42rpEuXBottNk1WUTn9EtqgP9+RsDb6FCIIwnGPL+qSC/R1+JxsX+YKd9b8RcBHWusa/7rXREpKjKXsJJ65JRVU19pIb8uWviAIQjPii+ivAQYqpdKVUlEYYV/o3EgpNRiIA1a72Ic7P3/z4KYEQ5Y9XFNEXxCE9opX0dda1wI3YVwzW4F3tdablVIPK6XOdWg6B1igtdaO2yul0jBPCsuD1WmvuCm2ZoVrtskYfUEQhBbAp9o7WuvPgc+dlj3g9PlBN9vuwc3Ab7PhwdKP6RBBQkyHFu2OIAhCWyE0RwzdWfqF5fRL6IySOvWCILRTQlP03cyalVlQLv58QRDaNaEp+i5mzaqsqSPvcIWIviAI7ZrQE/2qKqioaGTp7y06ita0/Rh9QRCEZiT0RN9NCYZjkTti6QuC0I4JXdF3svStksppIvqCILRjQk/03dTSt+bFjekQmjNECoIg+ELoib4bSz+rsEwGcQVBaPeEnui7sfSzCsvb5sQpgiAILUjoib4LS7+4vJriozX0l/ILgiC0c0JP9F3MmpUphdYEQRCAUBT9khLo2NH82amfF1dEXxCE9k3oib6LEgzH5sXt3qmVOiUIgtA2CD3Rd1GCIauwnJTunYhsq/PiCoIgtBChp4IuKmyayB1x7QiCIISe6Luopb//cCV94qJbqUOCIAhth9ATfSdLv7rWxuGKGnp0lolTBEEQQk/0nSz9ovIqAOK7RLVWjwRBENoMoSX6Wjey9AtLqwGIlykSBUEQQkz0S0vBZmtg6Rdalr6IviAIQoiJvosSDIWlluiLe0cQBCG0RN9FsbXCMnHvCIIgWISW6Luw9IvKqoiODKez1NEXBEEIMdF3UWytsKyKHuLaEQRBAEJN9F3Mj1tYVi2uHUEQBDuhKfpOlr6IviAIgiG0RL+4GJSC2NhjiwrLqkmQxCxBEAQg1ES/pAS6doUwc1p1Ns2h8iopwSAIgmAntETfqQRD8dFqbFpi9AVBECxCS/SdSjAUWTH6XcTSFwRBgFATfSdLv7BMSjAIgiA4Elqi71xsrUxKMAiCIDgSWqLfyNKXEgyCIAiOhJbou7D0I8IUsdGRrdgpQRCEtkPoiH5NDZSXN6qw2SMmCqVUK3ZMEASh7RA6ou+yBINk4wqCIDjik+grpWYppbYrpXYppe5xsX6eUmq9/W+HUqrEYV2KUmqRUmqrUmqLUioteN13ICYGPvwQZs06tqioXOruCIIgOOK13rBSKhx4DpgJ5ABrlFILtdZbrDZa69sc2t8MjHbYxZvAY1rrxUqpGMAWrM43IDoazj+/waLC0ioGJnZplsMJgiAcj/hi6U8AdmmtM7XW1cACYLaH9pcAbwMopYYCEVrrxQBa6zKt9dEA++wTWmt7hU0J1xQEQbDwRfT7APscPufYlzVCKZUKpAPf2BcNAkqUUh8qpdYppZ60Pzk0O6VVtVTX2cS9IwiC4ECwB3LnAO9rrevsnyOAKcCdwHigH3CV80ZKqeuUUmuVUmsLCgqC0pFjc+NKhU1BEIRj+CL6uUBfh8/J9mWumIPdtWMnB1hvdw3VAh8DY5w30lq/pLUep7Uel5CQ4FvPvWAlZkmFTUEQhHp8Ef01wEClVLpSKgoj7AudGymlBgNxwGqnbbsppSwlnw5scd62OSiSujuCIAiN8Cr6dgv9JuArYCvwrtZ6s1LqYaXUuQ5N5wALtNbaYds6jGtniVJqI6CAl4N5Au44VndH3DuCIAjH8BqyCaC1/hz43GnZA06fH3Sz7WJgZBP712QKyqpRCrp3EtEXBEGwCJ2MXCcKy6qI6xRFRHjInqIgCILfhKwiFpVVSYy+IAiCEyEr+iYxSwZxBUEQHAlh0a+ih4i+IAhCA0JW9IukBIMgCEIjQlL0K2vqKKuqFfeOIAiCEyEp+gX2EgwJIvqCIAgNCEnRLyq3l2AQ944gCEIDQlL0jxVbE0tfEAShAaEp+sdKMIjoC4IgOBLSot+js7h3BEEQHAlR0a+mS4cIOka2yHwtgiAIxw0hKvpV4toRBEFwQeiKvkTuCIIgNCIkRb+orFpmzBIEQXBBSIq+ce+IpS8IguBMyIl+TZ2N4qM1EqMvCILggpAT/eJj2bgi+oIgCM6EnOgXlFl1d8S9IwiC4EzIiX5hmbH0xb0jCILQmNATfam7IwiC4JaQE/2icnsJBnHvCIIgNCLkRL+wrJoOEWHEdIho7a4IgiC0OUJP9EuriI/pgFKqtbsiCILQ5gg90S+XuXEFQRDcEXqib7f0BUEQhMaEnuiXiegLgiC4I6RE31CrJO0AAAXZSURBVGbTFJVXS90dQRAEN4SU6B+uqKHOpqXCpiAIghtCSvRlblxBEATPhJToW3V3JHpHEATBNSEl+kVSd0cQBMEjISX6x9w7IvqCIAguCTnRDw9TdIuObO2uCIIgtElCSvTN3LhRhIVJCQZBEARXhJToF5ZVyYxZgiAIHggp0S8ok7o7giAInvBJ9JVSs5RS25VSu5RS97hYP08ptd7+t0MpVeKwrs5h3cJgdt6ZwtIqEsTSFwRBcIvXovNKqXDgOWAmkAOsUUot1FpvsdporW9zaH8zMNphFxVa61HB67JrtNYUlVfJ5CmCIAge8MXSnwDs0lpnaq2rgQXAbA/tLwHeDkbn/KG8uo7KGpuEawqCIHjAF9HvA+xz+JxjX9YIpVQqkA5847C4o1JqrVLqO6XUeU3uqRdqam2cMzKJIUldm+sQgiAIxz3BnlNwDvC+1rrOYVmq1jpXKdUP+EYptVFrvdtxI6XUdcB1ACkpKU06cFznKJ791ZgmdlsQBKF94Iulnwv0dficbF/mijk4uXa01rn210xgGQ39/Vabl7TW47TW4xISEnzokiAIgtAUfBH9NcBApVS6UioKI+yNonCUUoOBOGC1w7I4pVQH+/t4YDKwxXlbQRAEoWXw6t7RWtcqpW4CvgLCgde01puVUg8Da7XW1g1gDrBAa60dNh8CvKiUsmFuMH92jPoRBEEQWhbVUKNbn3Hjxum1a9e2djcEQRCOK5RSGVrrcd7ahVRGriAIguAZEX1BEIR2hIi+IAhCO0JEXxAEoR3R5gZylVIFwN4AdhEPFAapO8cTct7tCznv9oUv552qtfaa6NTmRD9QlFJrfRnBDjXkvNsXct7ti2Cet7h3BEEQ2hEi+oIgCO2IUBT9l1q7A62EnHf7Qs67fRG08w45n74gCILgnlC09AVBEAQ3hIzoe5vHN5RQSr2mlMpXSm1yWNZdKbVYKbXT/hrXmn0MNkqpvkqppUqpLUqpzUqpW+zLQ/28OyqlflBK/WQ/74fsy9OVUt/br/d37BVwQw6lVLhSap1S6jP75/Zy3nuUUhvtc4uvtS8LyrUeEqLvMI/vmcBQ4BKl1NDW7VWz8jowy2nZPcASrfVAYIn9cyhRC9yhtR4KnATcaP+OQ/28q4DpWusTgVHALKXUScBfgHla6wFAMXBNK/axObkF2Orwub2cN8A0rfUoh1DNoFzrISH6+D+P73GN1vpb4JDT4tnAG/b3bwDNNjVla6C13q+1/tH+vhQjBH0I/fPWWusy+8dI+58GpgPv25eH3HkDKKWSgbOBV+yfFe3gvD0QlGs9VETf53l8Q5ieWuv99vcHgJ6t2ZnmRCmVhpmB7XvawXnbXRzrgXxgMbAbKNFa19qbhOr1/nfgbsBm/9yD9nHeYG7si5RSGfbpZCFI13qw58gV2gBaa62UCsmwLKVUDPABcKvW+ogx/gyhet72OadHKaW6AR8Bg1u5S82OUuocIF9rnaGUmtra/WkFTrHPLZ4ILFZKbXNcGci1HiqWvj/z+IYqB5VSSQD21/xW7k/QUUpFYgT/31rrD+2LQ/68LbTWJcBSYBLQTSllGW2heL1PBs5VSu3BuGunA/8g9M8baDC3eD7mRj+BIF3roSL6Ps3jG+IsBK60v78S+KQV+xJ07P7cV4GtWuu/OawK9fNOsFv4KKWigZmY8YylwIX2ZiF33lrre7XWyVrrNMzv+Rut9aWE+HkDKKU6K6W6WO+BM4BNBOlaD5nkLKXUWRgfoDWP72Ot3KVmQyn1NjAVU3nvIPBH4GPgXSAFU6X0Iq2182DvcYtS6hRgBbCReh/vfRi/fiif90jMoF04xkh7V2v9sFKqH8YC7g6sAy7TWle1Xk+bD7t7506t9Tnt4bzt5/iR/WME8B+t9WNKqR4E4VoPGdEXBEEQvBMq7h1BEATBB0T0BUEQ2hEi+oIgCO0IEX1BEIR2hIi+IAhCO0JEXxAEoR0hoi8IgtCOENEXBEFoR/w/srZfXuq6iiAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(H.history['acc'])\n",
    "plt.plot(H.history['val_acc'], 'r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1a3c69e5c0>]"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD8CAYAAABw1c+bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8VNXZwPHfk52sLAlb2CGyCi4R3OqCXXC3SqtU39ZKi9a6VO1mfVvfLi5t1apoq1hcuojWjVqroqCACy6ArAICQSEhJCEs2bfJ8/5xZkiALDOTSSbMPN/PZz6Z3LnLucmd5577nHPPFVXFGGNM9IgJdwGMMcZ0LQv8xhgTZSzwG2NMlLHAb4wxUcYCvzHGRBkL/MYYE2Us8BtjTJSxwG+MMVHGAr8xxkSZuHAXoCWZmZk6bNiwcBfDGGOOGCtWrNitqln+zNstA/+wYcNYvnx5uIthjDFHDBH5wt95LdVjjDFRxgK/McZEGQv8xhgTZSzwG2NMlLHAb4wxUcYCvzHGRBkL/MYYE2UiJ/B7PHDnnbBgQbhLYowx3VrkBP7YWLjnHpg/P9wlMcaYbi1yAj9ATg5s2RLuUhhjTLcWWYF/1CjYvDncpTDGmG4tsgJ/Tg5s3w41NeEuiTHGdFuRF/hVIS8v3CUxxphuK/ICP1ie3xhj2hCZgd/y/MYY06rICvy9ekHv3hb4jTGmDZEV+MHV+i3wG2NMqyzwG2NMlInMwL9jB1RXh7skxhjTLbUb+EXkcREpFpF1bcxzhoisEpH1IrKk2fRpIrJJRLaIyM9DVeg2jRrlflqXTmOMaZE/Nf4ngWmtfSgiPYE/Axeo6njgG97pscDDwNnAOGCGiIzraIHbZT17jDGmTe0GflVdCuxpY5ZvAS+q6nbv/MXe6ZOBLaqap6p1wDPAhR0sb/ss8BtjTJtCkeM/CuglIotFZIWIfNs7PRvY0Wy+fO+0ztWzJ2RmWuA3xphWxIVoHccDZwE9gGUi8kGgKxGRWcAsgCFDhnSsRDZKpzHGtCoUNf58YIGqVqrqbmApMAkoAAY3m2+Qd1qLVHWOquaqam5WVlbHSmSjdBpjTKtCEfj/DZwqInEikgxMATYAHwM5IjJcRBKAy4CXQ7C99uXkQH4+VFV1yeaMMeZI0m6qR0TmAWcAmSKSD9wOxAOo6iOqukFEXgfWAI3AX1V1nXfZ64AFQCzwuKqu75S9OJSvgXfrVjj66C7ZpDHGHCnaDfyqOsOPef4I/LGF6a8CrwZXtA5o3rPHAr8xxhwk8u7chaabuKyB1xhjDhOZgT8jA7KyrIHXGGNaEJmBH2ywNmOMaYUFfmOMiTKRHfh37oTKynCXxBhjupXIDfy+Bt6tW8NbDmOM6WYiN/DbYG3GGNMiC/zGGBNlIjfwp6VBv34W+I0x5hCRG/jB5fntJi5jjDlIZAd+69JpjDGHifzAX1gIFRXhLokxxnQbkR/4wdI9xhjTTHQEfkv3GGPMAZEd+EeOdD+txm+MMQdEduBPS4P+/a3Gb4wxzUR24Afr2WOMMYewwG+MMVEm8gP/qFFQVATl5eEuiTHGdAuRH/itS6cxxhwkegK/pXuMMQaIhsDvG5ffAr8xxgDREPhTUmDgQAv8xhjj1W7gF5HHRaRYRNa18vkZIrJfRFZ5X79q9tlNIrJeRNaJyDwRSQpl4f02YQK8/robt8cYY6KcPzX+J4Fp7czzjqoe4339BkBEsoEbgFxVnQDEApd1pLBBu+ce16vnkkugtjYsRTDGmO6i3cCvqkuBPUGuPw7oISJxQDKwM8j1dMzRR8NTT8GyZXDDDWEpgjHGdBehyvGfJCKrReQ1ERkPoKoFwD3AdqAQ2K+qb4Roe4GbPh1uvRXmzIFHHw1bMYwxJtxCEfhXAkNVdRIwG5gPICK9gAuB4cBAIEVErmhtJSIyS0SWi8jykpKSEBSrBb/9LZx9Nlx/Pbz3XudswxhjurkOB35VLVPVCu/7V4F4EckEvgxsU9USVa0HXgRObmM9c1Q1V1Vzs7KygikH23ZXUlRW0/pMsbHw9NMwbJjL9xcUBLwdY4w50nU48ItIfxER7/vJ3nWW4lI8J4pIsvfzs4ANHd1eW772p6U8/u62tmfq2RPmz4fKSrj4Yqhp40RhjDERyJ/unPOAZcBoEckXkZkico2IXOOdZTqwTkRWAw8Cl6nzIfA8LhW01rutOZ2yF66cZKUlUlLuR6+dcePg73+Hjz5yaR9jjIkioqrhLsNhcnNzdfny5QEvd+HD75GeFMffZ07xb4Ef/hD+8hdX++/RI+DtGWNMdyEiK1Q11595I+rO3azUBP9q/D6nnQaqdlevMSaqRFbgT0tkd0Wd/wuMGeN+btzYOQUyxphuKKICf2ZqInsqa/E0+pm+OuooELHAb4yJKhEV+LPSEmlUKK30M93To4fr2rmhUzsbGWNMtxJZgT81EYDd5QGme6zGb4yJIhEV+DPTXOAvqQiggXfsWNi0CRobO6lUxhjTvURU4G+q8QcQ+MeMgepq2L69k0pljDHdS0QF/qBq/NazxxgTZSIq8KckxNIjPjawvvxjx7qfFviNMVEiogK/b9iG3YHU+DMzoU8fC/zGmKgRUYEfIDPQu3fBpXusS6cxJkpEXOAPuMYP1qXTGBNVIjLwB1zjHzsWiothT7BPmDTGmCNHxAX+zNRE9lbVU+8JoF++9ewxxkSRiAv8Wd4unaU2WJsxxrQo4gJ/pvcmroDSPcOGQWKiBX5jTFSIuMDvq/EH1MAbG+tG6rSePcaYKBB5gT+YGj90fc+eZ56B8eOhoqLrtmmMMURi4A9m2AZwgT8vD2oDXC5YL7wAn34Kjz/eNdszxhiviAv8SfGxpCXGBdels7ERtmzpnII1pwrvvefe338/eDydv01jjPGKuMAPbrC2oGr80DV5/i++gMJC+MpXYNs2eOmlzt+mMcZ4RWTgz0pNDGxoZnCNu9A1eX5fbf/uu2HkSLjnHncVYEx3t3Ej/Otf4S6F6aDIDPzB1PhTUmDo0K4L/KmpMGkS3HwzfPghvP9+52/XmI5Qhe9+Fy6/HOoCuE8mVLZtg6ee6vrtRqB2A7+IPC4ixSKyrpXPzxCR/SKyyvv6VbPPeorI8yKyUUQ2iMhJoSx8a4IaqA26brC299+HE0903UivvBJ694Z77+387RrTEUuWwAcfQEMDfPZZ12//979335eioq7fdoTxp8b/JDCtnXneUdVjvK/fNJv+APC6qo4BJgFd0lE+Ky2R8poGauoDbDT1denszMcwlpXB2rVwyinu9+Rk+MEPYP78rmlYNiZYd9wBSUnu/boW64Gda/Fi9/Ojj7p+2xGm3cCvqkuBgEcvE5EM4DRgrnc9daq6L+ASBiGom7jA9eypqoKCgk4oldcHH7gTiy/wA1x3HcTHw5/+1HnbNaYjPvoIFi6E//1fd6W6fn3Xbr+w0D0bG1xq1HRIqHL8J4nIahF5TUTGe6cNB0qAJ0TkExH5q4iktLYCEZklIstFZHlJSUmHCuMbtmF3IOP1QNeM2fP++xATA1OmNE3r39/lTZ94AkpLO2/bxgTrrrugVy+44QbIyen6Gv/Spe5naqoF/hAIReBfCQxV1UnAbGC+d3occBzwF1U9FqgEft7aSlR1jqrmqmpuVlZWhwp04CauYO7ehc7N87/3Hhx9NKSnHzz9llvcQ98feaTztm1MMNatc6nIG26AtDR3x3lX1/gXL3bfmcsuc1cfnZmOjQIdDvyqWqaqFd73rwLxIpIJ5AP5quo7PT+POxF0uqYaf4CBv29f6NkzuBr/ypXucrQtHo9L9TRP8/iMHw/TpsHs2V1397Ax/rj7btfr7YYb3O8TJrj2qOrqrivD4sVw6qlw8smuncyX9jFB6XDgF5H+IiLe95O96yxV1V3ADhEZ7Z31LODTjm7PH31SE4AgavwiLs8faOAvLYUvfQm+972251u71o3Nc/LJLX9+yy2ux8I//xnY9o3pLHl5MG+e64DQu7ebNn6869rZVWNbFRW5bZ1xRlOK1NI9HeJPd855wDJgtIjki8hMEblGRK7xzjIdWCciq4EHgctUD9yNdD3wTxFZAxwD3Bn6XThcYlwsPZPju65L55w5rlH4tdfcXbmt8d241VKNH+Css2DiRLjvPruhy3QPv/+963hw881N0yZMcD+7Ks+/ZIn7efrp7vuZnm6Bv4P86dUzQ1UHqGq8qg5S1bmq+oiqPuL9/CFVHa+qk1T1RFV9v9myq7x5+4mqepGq7u3MnWkuMzWIZ++CO7B27YJ9fnZAqqtz6Zljj3W///Wvrc/7/vswcKC7UawlIvDjH7v8aSDDOFRVQX29//Mb44+CAnjySbjqKhgwoGn6qFHuZNBVef4lS1yj7nHHuY4RJ5xggb+DIvLOXXDDNgRV4x871v30N4f4r3+53P4dd8DZZ8Pcue4Gl5a8955L87jMWMtmzHA1qp/8xL9cf0WFu0qYNcu/8hrjr/vuc+1SP/nJwdPj410Fqatq/IsXu1RqXJz7fcoUWLPGVXhMUCI28GemdaDGD/6le1Tdl2PsWPja1+Dqq91J4JVXDp+3oMClgVpL8/jExbl15uXBAw+0X4bbb4etW12vi9ZOOKZ7OJJGYd292/Uw+9a3YPjwwz/vqp49xcVu+PLTT2+aNmWK+1uuXNn5249QERv4g67xDx/uajT+NFwtXQqffAI33eQuQc85B7KzXc7/UL6xeNoL/OBG7Tz/fPjd79q+PX3FCjes8+jRLjW1bFn76zbh8fHHkJnpnsNwJHjwQddr59ZbW/58wgT4/PPOf5CQr//+GWc0TbMG3g6L3MCflkhlnYequgBrwXFx7gYVfwL/ffe5L/MVVzQtO3MmvP66+1I099570KMHHHOMf+W4916oqYHbbmv584YG+P73XRfUN95w2371Vf/Wbbreww+7k/N3vxuecW4C4fHAY4/Beec1pT4PNd57n+anndxRb/Fi15X0uGY9wfv1c+1kFviDFrGBP9PbpXN3eRCjCPrTpXPzZvjPf1w3tx49mqbPnOly+HPnHjz/++/D5MnuasIfOTlw/fXuCV2ffHL45/ff76bPng1Dhrg+zhb4u6eyMnjuOXcVFx8P3/hG985Pv/OO6+Bw+eWtz9NVPXuWLHHH9qHfmylTLPB3QMQG/qZHMNYEvvCYMe4GlbaGnn3gAXcwXnvtwdOHDGlq5PX1tKmqckHanzRPc7/8JfTpAz/60cHdO7dtg1/9ygWSSy5x08491zV47dgR2DZM53v2WXcM3Habu0dj7Vo3PlN39eyzbvDA885rfZ7hw92AbZ2Z5y8pcSeW5mken8mTYft2d4IyAYv8wB9MjX/8eHe5e/fdLd8avnevG1fnW99y4+wcatYs18j73/+63z/6yKVmWrtxqzU9e7o8/9KlTblhVXeVERvr0ge+HkLnnON+vvZaYNswnW/uXHdMTZ7s7s6+7TZ3/HTH5y03NLhj7bzzXIqlNbGxMG5c59b4ffn95g27Ppbn75DIDfypQT50HeDii+HSS12PmfPPP3zgNN8NWzfd1PLyvkbeRx91v/sadk8K4nEEM2e6sX1+8hOX8583DxYsgDvvhMGDm+YbO9blPS3d072sX++Cky8FCPB//wdTp8IPfwirV4d2ex298e/tt11N+9JL25+3s3v2LFnirjxycw//7Ljj3MnHAn9QIjbw905JQCSIYRsAEhNdgH34YTcU7XHHNR1g9fUur+67y7YlvkbeBQtcI+9777nake+W90DExbl8/uefu9TPj37kajuHpphE3Aln4UIb66c7mTvXpQR9HQDABax589zxMH067N8fmm39619uBM2ODCv+7LPuZqmzz25/3gkT3Lb8vdkxUIsXu/RoS+1iycnu+2eBPygRG/jjYmPonZwQXF9+cIH02mtd0I6JcTeQPPCAa6QrKDj4FvaW+Gp4jz3mavyBpnmamzoVLrzQPZt37153xREbe/h8554LlZVNl8gmvOrq4O9/hwsugENHnO3b1wXZbdvcsRKKITpmz3YnkZa6E/tb3hdfdMda8w4LrfH17OmMWv/u3a4tpKX8vs+UKa6b7JF0f0Q3EbGBH7zP3g2mxt9cbq67UeTss11te+ZM129+WjsPJfM18t5/v6sRBdqwe6h77nE1sV/8ovUrjTPPdFcrlu7pHv7zHxfAZs5s+fNTT3XtSC+8cHgvsEBt3gzvvuv+/3PmBPdM3IULXcXCnzQPdG7PnnfecT/bC/zl5V03WFwEifjAH3SNv7levdydsX/8o2v8+sUv3FVAe66+uqnbXkdq/ODGR9m1C37969bnSU52wd8Cf/cwd65r6/nqV1uf55Zb3POXf/vbjj3A/Mkn3TH50EPuOAlkrCefZ5+FjIy2y9vckCGuMtIZNf7Fi91VR0v5fR9r4A1aRAf+zGDv3m2JbwC1sjL49rf9W+bss2HQIHeZn5PT8TK01cvC55xz3A1C4X5+b1WVayQ80n38cdsjrrYmP9+18Vx5ZctpOR8R1zV3+3b429+CK6PHA0895YYNueoqGDHCnQACUVPjKjdf/7q7avCHiEv3dEaNf8kSd5WckND6PKNHuxOVBf6ARXTg96V6NJRDHPuT+/SJi3M5/gcfbHtgtlA691z3M1y1/i1bXPvHwIHufoi2hpzwR2mp69G0cGFoyheIzz5z6ZjJkwO/2/app1xX4Kuuan/eadNczfauu4Ibb2nhQtfu9N3vulr/tde6tM+aNf6vY8ECV6nxN83j0xk9e/bscWVvqRtnc76ROoN5+Pobb3T9U8S6E1Xtdq/jjz9eQ+HRJVt06M9e0bLqupCs74gxerTq177WddvzeFT/+1/Vs89WBdW4ONXp01UTElS/9a3g1tnYqPrPf6pmZbl1pqerbt0a2nK3t/2pU912s7JUBw9W/fxz/5b1eFRHjFA980z/t/fvf7v9fOqpwMt66aWqvXur1tS430tLVZOSVGfN8n8dM2ao9umjWhfgd+W++1y5i4sDW64tL73k1vnOO+3Pe9ttqrGxqpWV/q//44/dMgMHqu7dG3w5uxlgufoZY8Me5Ft6hSrwv7hyhw792Su6tbg8JOs7Ytx0k2piompFRejWWVSkumyZ6quvuoD80EOqv/2t6o9+5IIcqA4YoPrrX6vu3OmWuf12N/2NNwLb1rZtqtOmuWUnT1b9z39UMzJUc3Obgltn+/vf3fYfflh11SrVnj1VR45s2re2vPWWW/Yf//B/e42NqpMmqR51lGpDg//L7dnj/tfXXXfw9KuuUk1O9i+wVVaqpqSofv/7/m/XZ8ECt69vvx34sj5lZarr16u+/rrqY4+pnnWWao8e/v2vX37ZbX/pUv+2VVOjOmGCO5nHxKh+73vBl7ubscDv9c5nJTr0Z6/oh3mlIVnfEePNN92/9j//Cc365s51wcV1Ojz4lZKietppqs8+e3htsbpaNSdHddQo97499fWq997rAlZqquqDDzYFwRdfdNu78cb211NXp7prV+D76bNnjwsMkyc3bX/ZMrev48erlpS0vfwVV7gTVVVVYNt9/nm3j08/7f8yDz/sllmx4uDpK1a46fff3/46nnvOzbtwYWDlVVUtKHDLzp4d2HL19arf/Kb7Ox16TImoXnmlf+spKnLL/PGP/s3/y1+6+V95RfWnP3Xv33wzsLJ3Uxb4vTYU7tehP3tFX1ntRy0tktTUuMB5zTUdW091tasFguqXv+y+LO+/r7phgwustbXtr2PRIrf8L3/Z9nxffOFq9KB63nmq27cfPs/117vPX3qp9fVs26Z63HEuzfTCC+2XryWzZrna4CefHDx90SJ3Ajz+eNV9+1pedu9el2b5wQ8C367H404s48a59/444QTViRPdFcOhTjrJnXjbW9f06ap9+7pgHKjGRnc1FOix5ksR/c//qN59t7uKXLrU/f/8Oa6aGzbM7UN7Vq50KZ5vf9v9XlXl/j7DhqmWH/lZAQv8XrvLa3Toz17RJ97NC8n6jigXXaQ6ZEjLAcEfn3/eFIhvvTWw9MOhrrhCNT7enTBasnq1y7dmZLgrh9bKXFPjgm7Pni5AHOq111yuOyND9ZhjXPCeMyewsr73ntvnm25q+fNXXnFtGKee6oLFli3uauTXv1a95BIXREB1+fLAtuszb55b/rnn2p933To375/+1PLn//iH+3zBgtbXUV7u0irXXhtceVXd3+LUU/2ff9s2d1V37rnBH5/NXXqpa4NpS22tS6UNGOCu6HzeecddYdxwQ9vL79qlOn++/yfkMLDA7+XxNOqIW/+rf3x9Y0jWd0SZM8f9e9etC3zZN95wDX3p6e5g76iiIhesTz/98C/6W2+57WRnq65Z0/66tmxx80+e3FQz9Hhc4BVxtd/Nm137hq+x+Y47/AswdXUu/ztoUNs1wGefdSeV2Fg9KD2Rk+OC/2OPBR/QGhpc4/zEie0HmVtucSeh1hpWa2pcyuqCC1pfx9NPu/IvWRJceVVVr75atVcv//a5sdH9X1JS/G8sb4/v6qGtk+3//Z+b59//Pvyz665z/79332152ZdfbupkMHVqy1ej3YAF/mZO+N2b+tPnVodsfUeMHTvcv/cPf/B/mYoK12Ar4lIOmzaFrjyPPurK8+STTdOeecalZMaPD+zL5MtJ33yz68Fyzjnu9yuuOLh3R12dm+ZrG2gvkP7hD9puKsnn5ZfdOh97TPXDD0PbkP63v7lytHXSratz6ZmLLmp7Xbfd5k5SLQXZxkaXVhs4sGM12dmzXXn9afh+5pm2r1KCsWWLq1jExLiUU1HRwZ+vWuVOkJdf3vLy5eWqQ4e6hvXm7TIVFe6kBu4K8g9/cCesjAyXmupmLPA3c84DS/WqJz4K2fqOKJMmuZ4oDz2kunZty1/uqiqXC7/0Unf5Da5rXygDmarb9sknq2Zmqu7e3VRLO+20gy+9/XXttXqgJ1F8vGvkbKnG6fG4nkfgupa2lj/+/HO3/23VjrtKfb37vx13XOu1aF/3z5ZqsM1t3+4C4s9/7n73eFx648YbXXoEVH/yk46V19eLqb3eW3v2uJNVbm7HUoctKSlxbUCxsappaa7doLranSCPPVa1Xz933LXmjTfcPvzsZ+735cvdlZeI+/v4ehht2eLaTkD1sssOP3YbG92V6733umPpK19xJ9dLLnEnnquucsfuT3+qeuedqn/+s7vqeu0114GgA5UtC/zNfHvuh3r+bD/6A0eiF19s+nKDS998/euqDzzgas2XX+4agcEF5Kuvdt3yQpF3bcmaNa7mNXKk2+b06f719mlJdbULINnZ7gvTlsZG1bvucts84wzXzfR3v1P9/e/dF/TBB9305GTXyNwdzJ3ryvvMMy3/Py66yAVRf/rdf/3r7n//gx+o9u/v1puY6ALTU08F3ph6qOJi/2rx3/++C8wrV3Zse23ZsEH1/PNdeYYOdcEZ3HehPTNnuvLdcIM7TrOzXYP+oerr3fHjm+fFF92V7OWXuxOM7/uWk6N64onuxDNunDvus7Pddy0pqWm+5q+srKB3PaSBH3gcKAbWtfL5GcB+YJX39atDPo8FPgFe8bdQoQz8t/xrlZ54ZxDd1CJFY6NqXp7qE0+ofuc7TY2P4PKyM2e62k4wPTqC4etCd/31Ha/11dQE1q//sceaTnQtdSEMtEtiZ6qrc91gfVc1V1zhgsuOHS6VERfncvz+ePttt57kZHeynTfP9Z0Ppawsdyy1ZsmS0Fxd+GvRInfF66uZ+2PvXpf2AtVvfMOlEdvy8cfuqqB50J4xQ/Xxx/1LXVZXqxYWqn76qest9+qr7V/BtSGQwC9u/taJyGlABfA3VZ3QwudnAD9W1Raf0yYiNwO5QHpr8xwqNzdXly9f7s+s7fr96xv56zt5fPa7s5GuGjahu/viC/eIxilT/H8GcKh4PO7hI8ce23XDWByqsdENjVBf714NDe72/2Cel9CZSkrcCJ8LF8KiRVBc7KZnZrpRP9etaxoauT1r18LIkW4gv84wdSpUV8OyZYd/VlsLkya5QejWrvVvzKlQ8Hjc3+2UU/zf5po17tkX55/v3/FZVQUvv+yGJ5k40b/BGzuJiKxQ1TZGtWsS194MqrpURIYFWZBBwLnAHUA7A9h3jszUROo9yv7qenomtzHgUzQZOtS9wiE21j3YJpxiYtzgX20NANYdZGW5sX6uusrVKdetcyeBhQvdIz/9DfrgnuLWmcaPd+MTqbqAqepOsB4P3HEHbNoEr7/edUEf3LHm70ijPhMntj7seUuSk+GyywLbRjfQbuD300kishrYiav9+0Y/uh/4KZDW3gpEZBYwC2DIkCEhKlbTs3d3V9Ra4DdHLhEXvI8+uvVHfobThAlubPz4eBfwD80kXH65Gz3UdAuhCPwrgaGqWiEi5wDzgRwROQ8oVtUV3nRQm1R1DjAHXKonBOUCIDPVBfvi8lpG9W33/GOMCcY3v+meA1Bf766oYmObfqakuJFDTbfR4cCvqmXN3r8qIn8WkUzgFOAC78kgCUgXkX+o6hWtrasz9PXW+EM2Lr8x5nC9esHtt4e7FMZPHW6JEJH+4m01FZHJ3nWWquqtqjpIVYcBlwFvdXXQB8hKTQJgd0UHnm5kjDERpN0av4jMw3XZzBSRfOB2IB5AVR8BpgM/EJEGoBq4TNvrKtSF0nvEkRAbYzV+Y4zx8qdXz4x2Pn8IaPM5b6q6GFgcSMFCRUTITE0IzbN3jTEmAkT0oxd9stISKbYavzHGAFES+If2SWFrcUW4i2GMMd1CVAT+CdnpFOyrZm+lNfAaY0x0BP6BGQCs31nWzpzGGBP5oiLwj/cG/nU794e5JMYYE35REfgzkuMZ3LsH6wos8BtjTFQEfnDpHkv1GGNMNAX+7Ay27a6kvKY+3EUxxpiwiprAP35gOgCfWq3fGBPloijw+xp4LfAbY6Jb1AT+rLRE+qcnsd4aeI0xUS5qAj+4G7msS6cxJtpFVeAfPzCDLcUVVNd5wl0UY4wJm6gK/BOyM2hU2LDL8vzGmOgVZYHf9eyxPL8xJppFVeDvn55E75QE1hVYjd8YE72iKvCLCOMHWgOvMSa6RVXgB5fn/6wkMknjAAAVkElEQVSonNoGa+A1xkSn6Av8AzOo9yibi+zBLMaY6BR9gd/bwGsjdRpjolXUBf4hvZNJS4qzPL8xJmpFXeA/0MBrPXuMMVEq6gI/uDz/hsIyGjyN4S6KMcZ0uXYDv4g8LiLFIrKulc/PEJH9IrLK+/qVd/pgEXlbRD4VkfUicmOoCx+sCdkZ1DY0srWkMtxFMcaYLudPjf9JYFo787yjqsd4X7/xTmsAblHVccCJwA9FZFzwRQ0da+A1xkSzdgO/qi4F9gS6YlUtVNWV3vflwAYgO+ASdoLhman0iI+1Bl5jTFQKVY7/JBFZLSKvicj4Qz8UkWHAscCHra1ARGaJyHIRWV5SUhKiYrUsNkYYNzDdavzGmKgUisC/EhiqqpOA2cD85h+KSCrwAvAjVW21K42qzlHVXFXNzcrKCkGx2jZhYDrrd5bR2Kidvi1jjOlOOhz4VbVMVSu8718F4kUkE0BE4nFB/5+q+mJHtxVK47MzqKrzsK3UGniNMdGlw4FfRPqLiHjfT/aus9Q7bS6wQVXv6+h2Qm2C7xm8lu4xxkQZf7pzzgOWAaNFJF9EZorINSJyjXeW6cA6EVkNPAhcpqoKnAL8DzC1WVfPczppPwKW0y+VhNgY1tvD140xUSauvRlUdUY7nz8EPNTC9HcBCb5onSs+NoYxA9Ksxm+MiTpReeeuz/iBGawt2E+93cFrjIkiUR34zxrTl/KaBpZs6tzuo8YY051EdeA/fXQWmakJPL8iP9xFMcaYLhPVgT8+NoaLjslm0cYi9lTWhbs4xhjTJaI68ANccvwg6j3Ky6sKwl0UY4zpElEf+McOSGdCdjrPWbrHGBMloj7wA3zj+MGs31nGp9an3xgTBSzwAxdMGkh8rPDCSqv1G2MinwV+oFdKAl8e24/5nxRYn35jTMSzwO81/fhBlFbWsdj69BtjIpwFfq/Tj8oiMzWR51fsCHdRjDGmU1ng94qLjeHi47JZtKGY0oracBfHGGM6jQX+Zi45bhANjcrLq3eGuyjGGNNpLPA3M7p/GhMHZfDccuvdY4yJXBb4DzH9+EF8WljGensQuzEmQlngP8T5EweSEBvDCytsCAdjTGSywH+IXikJfHlcX+avKqCuwfr0G2MijwX+Flw+ZSh7Kuu4/eV1uKdIGmNM5LDA34JTRmVy3ZmjmPfRDuYszQt3cYwxJqTafeZutLr5K0fxeWkld7++kaF9kpk2YUC4i2SMMSFhNf5WxMQI93xjEscM7smPnl3F6h37wl0kY4wJCQv8bUiKj+Wxb+eSmZrI9/62nIJ91eEukjHGdJgF/nZkpibyxJUnUFPvYeaTH1NeUx/uIhljTIe0G/hF5HERKRaRda18foaI7BeRVd7Xr5p9Nk1ENonIFhH5eSgL3pVy+qXxl8uPZ3NxBdc9/QkNNnSzMeYI5k+N/0lgWjvzvKOqx3hfvwEQkVjgYeBsYBwwQ0TGdaSw4XRqTia/u2gCSz4r4ZuPLmNzUXm4i2SMMUFpN/Cr6lJgTxDrngxsUdU8Va0DngEuDGI93caMyUN44LJj2La7knMffJfZizbbTV7GmCNOqHL8J4nIahF5TUTGe6dlA80Ht8/3TmuRiMwSkeUisrykpPs+DOXCY7J58+bT+dqE/tz75mdc8NC7rMm3Hj/GmCNHKAL/SmCoqk4CZgPzg1mJqs5R1VxVzc3KygpBsTpPZmois2ccy2PfzmVvVR0XPfwed726geo6T7iLZowx7epw4FfVMlWt8L5/FYgXkUygABjcbNZB3mkR4yvj+vHGTadz6QmDeXRpHqf98W0ef3cbNfV2AjDGdF8dDvwi0l9ExPt+snedpcDHQI6IDBeRBOAy4OWObq+7yegRz10XT+S5a05iVFYqv3nlU770h7eZaycAY0w3Je0NQiYi84AzgEygCLgdiAdQ1UdE5DrgB0ADUA3crKrve5c9B7gfiAUeV9U7/ClUbm6uLl++PJj9CbsP8kp5YOFmluWVkpWWyDWnj+TyKUNIio8Nd9GMMRFMRFaoaq5f83bH0SeP5MDv82FeKQ8s2sz7W0vJTE1k1mnDuXzKUFISbXgkY0zoWeDvRj7atofZb23mnc276Z2SwMxTh/Ptk4aSlhQf7qIZYyKIBf5uaMUXe5n91mYWbyoho0c8V50ynCtPGUZGDzsBGGM6zgJ/N7Ymfx8PLtrCwg1FpCXFcd2Zo/jOycOsDcAY0yEW+I8A63fu554Fm3h7UwnZPXvw02mjOX/iQGJiJNxFM8YcgQIJ/DY6Z5iMH5jBE9+dzD+/N4WMHvHc+MwqLnz4PZZtLQ130YwxEc4Cf5idMiqTV64/lXu/MYndFbXMeOwDZj75MUs+K/F7FNDishr2V9lw0cYY/1jfwm4gJka45PhBnDtxAHPf3cajS7ayaGMxmamJXDBpIBcfl834gel475PD06is2rGPtzYW8dbGEjYUlhEbIxw/tBdnjenLWWP7MTIr5cD8xhjTnOX4u6HaBg9vbyxh/icFvLWxmDpPIyOzUjj36AHk761m8Wcl7KmsIzZGyB3aizPH9KWytoFFG4r5tLAMgKF9kpk6pi/jBqRTUdtAWXUDZTX1lNfUU1bdQEJcDN/70nAmDuoZ5r01xoSCNe5GkP1V9by6rpCXPingo2176Jkcz5mj+zJ1TF9Oy8kiI/ng7qA791Xz1sZiFm0o4r2tpQcNG52SEEtaUjzpPeIoLq9lX1U9500cwI+/OpphmSldvWvGmBCywB+h9lXVkZYUT6yfPX+q6zyUlNeSlhRHWlIccbFNTTrlNfU8tjSPx97ZRr2nkW9NGcL1U3PISkvsrOKHlKdRyd9bRYwIg3r1sLSWiXoW+I3fistqePCtzcz7aAeJcTFcefIwMlMTvWmhBsqq6ymrqaeitoGE2BgyesQfeKV7X3UNjRSV1bBrfw27mv2sbWikf3oSAzLcq39GDwb2TGJARg+GZyYzuHcyiXFt379QXlPP9j1V5JVUsqW4gi0lFWwtriBvd+WBq5meyfFMGJjBhOwMJg7K4OjsjG51Mqht8PDc8nwWrN/FV8f355u5g9rdb2MCZYHfBCyvpIJ73tjEq2t3HZiWkhDrgntSPKlJcdQ2eCirbmC/92TQ/NCJEeiblkS/jCT6pyfSPz2JxPhYdxLYX8PO/dUUldVQ79GDlhncO5nhmSkMz0xhUK9k9lbW8cWeKrbvqWLHnir2VNYdmF8EBvdKZlTfVEb1TWVkVgoNjcq6gv2syd/PZ0XlB9afnhTH6P5p5PRLY3S/NI7ql8ZR/VLpk5pITb2HorIaCr1lK9xfQ0l5LaP6pnLiiN4MzwxNw3hNvYfnlu/gz4u3Uri/hr5piRSX19I/PYmrTx/BjMnBDd63aVc523ZXcNpRWSQnhLZ/RoOnkf+uLeQvi7fyRWnVgavFtKR40pLiSE+KJystkaOzM5g0OIPhmal+X4F21Oe7K9laUsGUEX1ItTGvDmOB3wRtd0UtcTFCauLBqaFDNTYq5bXuiiA+NobM1IQ25/ctU1pZR/7eKj4vrWRbSSV5uyvJK6lk2+5Kqus9xMYI2T17MKR3MkP6JDOkdzKDeyUzIsudHNoKlLUNHjbtKmdtwX7W7yxjc1E5m3aVU1bTcGCelIRYKlt4YE5SfAw19e4Kom9aIieO6ON99aZPaiLVdR6q6hqoqvN4Xw3ExghZaYlkpibSKznhQACsqffwzEfbeWRJHrvKasgd2osbv5zDqaMyeX9rKQ8u2syH2/aQmZrI1aeN4PITh/gVwGvqPdy/cDOPvZOHp1FJTojl7AkDuPi4bE4c0adDAbi2wcOLKwt4ZIkL+Ef1S+XUUVlU1jZQXuu9+qtpoLymnl37a6jy/g1TEmKZkJ3BpME93clgUE8G927/aqve08gXpZWk94inb1pSm+VasL6IeR9uZ1meu8clIS6G03Iy+dr4/nx5bD96pSQEvd+RxAK/OeKoKnsq68joEd/uCSTQ9RaX17JpVzmfFZWTv7eazNQE+mf08KafkuifnkRyQiyfl1axbGspH+S5V3F5rd/biRHok+pOArsraikpr2XysN7c+OUcTh7Z57BA+GFeKbPf2sK7W9zgfZeeMJjpxw9iZFZqi+v/IK+UW19cy7bdlXwzdxDnTRzIf9cU8uraQsprGxiQkcRFx2ZzwaSBjO6X5vcd4NV1HuZ9tJ05S91JauKgDH545ii+MrZfq+vwNCpbSypYk7+fNfn7WJ2/nw07y6jzNKXejvam3SYO6snY/ukU7q/m08IyNhSW8WlhGZ8VVRxI1Q3MSGLS4J5MHNSTSYNdqq6orJZnPtrOCyvz2VtVz6BePZgxeQgTB2Xw9sYSFqzfRcG+amJjhBNH9GbahAFceMxA0jtp8MMP8kr575pCeiTEkproroLcT9dZYmifFAakJwV9531jo7L8i71sLalgxuQhQa3DAr8xHaSqbNtdyYfb9lBZ20ByQhzJCbH0SIgl2ftq8Ci7K+oOBPrdFe4VI8KVpwzjpBGHB/xDrfhiL39ZvIW3N5XgaVSOG9KT6ccP5rxJA0hPiqespp67X9vI0x9uZ0jvZO66+GhOGZV5YPmaeg9vflrEiyvzWbp5N55GJTUxjgnZ6UwcdHAtfE9lHZt2lbNxVzkbd5V5T4YVVNd7mDy8N9edOYov5WQGleaqa2jks6Lyg04GnxWV42k8OL70Tklg3IB0xg1MZ3S/NPZW1bE6fz+rd+xj+54qwKX0VCEuRvjq+H7MmDyEU0ZmHhRUVZV1BWW8vr6Q19btIq+kkuSEWKYfP4jvnDys1RNooIrLa7jzvxuYv2onyQmxqEJ1Kw9YSoqPYXhmKiMyUxiR5V5H9UtjVN/UVtt0thRX8NIn+cz/ZCcF+6rpnZLAB7eeRUJc4JUfC/zGHGGKy2qYv6qA55bns7m4gsS4GL48rh8rPt9LcXkNV50ynJu/elSbKaGS8lre3lTMmvx9rM3fz4bC8gO18MS4GGqbde3tk5LAmAFpjO6XztlH9+eEYb1Dvk819R7W73QnmAE9kxg3IJ2+aYmtnlj2VNa5k8aO/SQnxHLRsdl+9zJbm7+fJ97fxiurC6nzNHL6UVl895RhnJaTRUyMeFNLVWwtqSCvpJK8kgqy0hKZOqYvxw7pdViarMHTyD8++IJ73/iM2oZGrjl9BNeeOYqk+FjqPY1U1DS4+2Nq6tlXVc/npZUH1pu3u5Ide6rwnfPiYoSRWamMHZDGmAHpjOmfRl5JJS99UsDagv3ECHwpJ4uvH5vNV8f3C7rdxgK/MUcoVWVN/n6eX5HPy6t3MrBnD+6++GgmDQ78RrvmtfDNxeVk9+zB6P5pjOmffsR02w1USXktT3+4nX98+AUl5bUM7ZNMbIywvbSKhmZXH5mpieytqsPTqPRKjueM0X05c0xfTs/JYuvuCv73pXV8WljGl3Iy+c2FExge4H0utQ0eviitYtOucjYUlrHR+7Nwf82BeY7OzuCiY7M5f9KANts5/GWB35gIoKrdpkvqkaauoZFX1xbywsp8UhLiGNk3hZFZqYzMSmV4VgrpSfHsr65n6WclvLWxmMWbitlbVU9sjOBpVPqlJ/Kr88ZzztH9Q/o/2FdVx8Zd5WSmJjKqb2jSUT4W+I0xJgBu/Ku9vLWxmPjYGL73pRFHXJfRQAL/kbVnxhjTCdwgh705fmjo2zq6IxuW2RhjoowFfmOMiTIW+I0xJsr4FfhF5HERKRaRde3Md4KINIjI9GbT/iAi60Vkg4g8KNZNwRhjwsrfGv+TwLS2ZhCRWOD3wBvNpp0MnAJMBCYAJwCnB1NQY4wxoeFX4FfVpcCedma7HngBKG6+KJAEJACJQDxQFHgxjTHGhEpIcvwikg18HfhL8+mqugx4Gyj0vhao6oZW1jFLRJaLyPKSkpJQFMsYY0wLQtW4ez/wM1VtbD5RREYBY4FBQDYwVUS+1NIKVHWOquaqam5WVlaIimWMMeZQobqBKxd4xttumwmcIyINQA7wgapWAIjIa8BJwDttrWzFihW7ReSLIMuSCewOctkjme13dLH9ji7+7PdQf1cWksCvqsN970XkSeAVVZ0vIpcC3xeRuwDBNeze78f6gq7yi8hyf29bjiS239HF9ju6hHq//Qr8IjIPOAPIFJF84HZcQy2q+kgbiz4PTAXW4hp6X1fV/3SkwMYYYzrGr8CvqjP8XaGqXtnsvQe4OvBiGWOM6SyReOfunHAXIExsv6OL7Xd0Cel+d8thmY0xxnSeSKzxG2OMaUPEBH4RmSYim0Rki4j8PNzl6UwtjZ0kIr1F5E0R2ez92SucZQw1ERksIm+LyKfesZ9u9E6P6P0GEJEkEflIRFZ79/3X3unDReRD7zH/rIgkhLusoSYisSLyiYi84v094vcZQEQ+F5G1IrJKRJZ7p4XsWI+IwO8dJ+hh4GxgHDBDRMaFt1Sd6kkOHzvp58AiVc0BFnl/jyQNwC2qOg44Efih938c6fsNUAtMVdVJwDHANBE5ETc21p9UdRSwF5gZxjJ2lhuB5nf7R8M++5ypqsc068YZsmM9IgI/MBnYoqp5qloHPANcGOYydZpWxk66EHjK+/4p4KIuLVQnU9VCVV3pfV+OCwbZRPh+A6hT4f013vtSXFfp573TI27fRWQQcC7wV+/vQoTvcztCdqxHSuDPBnY0+z3fOy2a9FPVQu/7XUC/cBamM4nIMOBY4EOiZL+9KY9VuEEQ3wS2AvtUtcE7SyQe8/cDPwV8Q8H0IfL32UeBN0RkhYjM8k4L2bFuz9yNQKqqIhKR3bVEJBU3CuyPVLWs+eMdInm/vffEHCMiPYGXgDFhLlKnEpHzgGJVXSEiZ4S7PGFwqqoWiEhf4E0R2dj8w44e65FS4y8ABjf7fZB3WjQpEpEBAN6fxe3Mf8QRkXhc0P+nqr7onRzx+92cqu7DjXh7EtBTRHyVt0g75k8BLhCRz3Gp26nAA0T2Ph+gqgXen8W4E/1kQnisR0rg/xjI8bb4JwCXAS+HuUxd7WXgO9733wH+HcayhJw3vzsX2KCq9zX7KKL3G0BEsrw1fUSkB/AVXBvH24DvaXcRte+qequqDlLVYbjv81uqejkRvM8+IpIiImm+98BXgXWE8FiPmBu4ROQcXE4wFnhcVe8Ic5E6TfOxk3APtrkdmA/8CxgCfAF8U1Xbe3jOEUNETsWN6rqWppzvL3B5/ojdbwARmYhrzIvFVdb+paq/EZERuNpwb+AT4ApVrQ1fSTuHN9XzY1U9Lxr22buPL3l/jQOeVtU7RKQPITrWIybwG2OM8U+kpHqMMcb4yQK/McZEGQv8xhgTZSzwG2NMlLHAb4wxUcYCvzHGRBkL/MYYE2Us8BtjTJT5f9Q0YmT/Jok0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(H.history['loss'])\n",
    "plt.plot(H.history['val_loss'], 'r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_cat = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = np.argmax(y_pred_cat, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.76      1.00      0.86      7436\n",
      "           1       0.00      0.00      0.00      2333\n",
      "\n",
      "   micro avg       0.76      0.76      0.76      9769\n",
      "   macro avg       0.38      0.50      0.43      9769\n",
      "weighted avg       0.58      0.76      0.66      9769\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test,y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensorflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### Create the Feature Columns for tf.esitmator\n",
    "\n",
    "** Take note of categorical vs continuous values! **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Import Tensorflow **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "** Create the tf.feature_columns for the categorical values. Use vocabulary lists or just use hash buckets. **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 330,
   "metadata": {},
   "outputs": [],
   "source": [
    "workclass = tf.feature_column.categorical_column_with_hash_bucket('workclass',hash_bucket_size=1000)\n",
    "education = tf.feature_column.categorical_column_with_hash_bucket('education',hash_bucket_size=1000)\n",
    "marital_status = tf.feature_column.categorical_column_with_hash_bucket('marital_status',hash_bucket_size=1000)\n",
    "occupation = tf.feature_column.categorical_column_with_hash_bucket('occupation',hash_bucket_size=1000)\n",
    "relationship = tf.feature_column.categorical_column_with_hash_bucket('relationship',hash_bucket_size=1000)\n",
    "race = tf.feature_column.categorical_column_with_hash_bucket('race',hash_bucket_size=1000)\n",
    "gender = tf.feature_column.categorical_column_with_hash_bucket('gender',hash_bucket_size=10)\n",
    "native_country = tf.feature_column.categorical_column_with_hash_bucket('native_country',hash_bucket_size=1000)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Create the continuous feature_columns for the continuous values using numeric_column **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 331,
   "metadata": {},
   "outputs": [],
   "source": [
    "age = tf.feature_column.numeric_column('age')\n",
    "education_num = tf.feature_column.numeric_column('education_num')\n",
    "capital_gain = tf.feature_column.numeric_column('capital_gain')\n",
    "capital_loss = tf.feature_column.numeric_column('capital_loss')\n",
    "hours_per_week = tf.feature_column.numeric_column('hours_per_week')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Put all these variables into a single list with the variable name feat_cols **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 332,
   "metadata": {},
   "outputs": [],
   "source": [
    "feat_cols = [age, workclass, education, education_num, marital_status,\n",
    "       occupation, relationship, race, gender, capital_gain,\n",
    "       capital_loss, hours_per_week, native_country]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Input Function\n",
    "\n",
    "** Batch_size is up to you. But do make sure to shuffle!**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 333,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_func = tf.estimator.inputs.pandas_input_fn(x=X_train,y=y_train,batch_size=100,num_epochs=None,shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create your model with tf.estimator\n",
    "\n",
    "**Create a LinearClassifier.(If you want to use a DNNClassifier, keep in mind you'll need to create embedded columns out of the cateogrical feature that use strings, check out the previous lecture on this for more info.)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 334,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "WARNING:tensorflow:Using temporary folder as model directory: /var/folders/q_/7b87rxq97d74fk63y0b6xcy80000gn/T/tmpa5etbq7o\n",
      "INFO:tensorflow:Using config: {'_model_dir': '/var/folders/q_/7b87rxq97d74fk63y0b6xcy80000gn/T/tmpa5etbq7o', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x1a1fe35f98>, '_task_type': 'worker', '_task_id': 0, '_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
     ]
    }
   ],
   "source": [
    "model = tf.estimator.LinearClassifier(feature_columns=feat_cols,n_classes=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Train your model on the data, for at least 5000 steps. **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 335,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Saving checkpoints for 1 into /var/folders/q_/7b87rxq97d74fk63y0b6xcy80000gn/T/tmpa5etbq7o/model.ckpt.\n",
      "INFO:tensorflow:loss = 69.31474, step = 1\n",
      "INFO:tensorflow:global_step/sec: 192.433\n",
      "INFO:tensorflow:loss = 0.00035486039, step = 101 (0.521 sec)\n",
      "INFO:tensorflow:global_step/sec: 264.805\n",
      "INFO:tensorflow:loss = 3.6905472e-05, step = 201 (0.379 sec)\n",
      "INFO:tensorflow:global_step/sec: 292.112\n",
      "INFO:tensorflow:loss = 6.332671e-05, step = 301 (0.343 sec)\n",
      "INFO:tensorflow:global_step/sec: 299.829\n",
      "INFO:tensorflow:loss = 0.00060355786, step = 401 (0.334 sec)\n",
      "INFO:tensorflow:global_step/sec: 344.996\n",
      "INFO:tensorflow:loss = 0.0002679027, step = 501 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 261.138\n",
      "INFO:tensorflow:loss = 5.5205506e-05, step = 601 (0.381 sec)\n",
      "INFO:tensorflow:global_step/sec: 262.962\n",
      "INFO:tensorflow:loss = 0.00017909532, step = 701 (0.381 sec)\n",
      "INFO:tensorflow:global_step/sec: 327.915\n",
      "INFO:tensorflow:loss = 0.00012366571, step = 801 (0.306 sec)\n",
      "INFO:tensorflow:global_step/sec: 262.759\n",
      "INFO:tensorflow:loss = 0.00010506217, step = 901 (0.379 sec)\n",
      "INFO:tensorflow:global_step/sec: 347.101\n",
      "INFO:tensorflow:loss = 0.0001078564, step = 1001 (0.290 sec)\n",
      "INFO:tensorflow:global_step/sec: 315.488\n",
      "INFO:tensorflow:loss = 5.261989e-06, step = 1101 (0.317 sec)\n",
      "INFO:tensorflow:global_step/sec: 265.971\n",
      "INFO:tensorflow:loss = 0.00035508056, step = 1201 (0.377 sec)\n",
      "INFO:tensorflow:global_step/sec: 263.774\n",
      "INFO:tensorflow:loss = 4.8439048e-05, step = 1301 (0.378 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.856\n",
      "INFO:tensorflow:loss = 7.299482e-05, step = 1401 (0.310 sec)\n",
      "INFO:tensorflow:global_step/sec: 351.179\n",
      "INFO:tensorflow:loss = 6.892047e-05, step = 1501 (0.285 sec)\n",
      "INFO:tensorflow:global_step/sec: 322.813\n",
      "INFO:tensorflow:loss = 0.00010204307, step = 1601 (0.312 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.598\n",
      "INFO:tensorflow:loss = 0.0003385248, step = 1701 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 365.748\n",
      "INFO:tensorflow:loss = 7.412107e-06, step = 1801 (0.274 sec)\n",
      "INFO:tensorflow:global_step/sec: 329.161\n",
      "INFO:tensorflow:loss = 2.526549e-05, step = 1901 (0.303 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.873\n",
      "INFO:tensorflow:loss = 0.0001893299, step = 2001 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.53\n",
      "INFO:tensorflow:loss = 1.5307729e-05, step = 2101 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.846\n",
      "INFO:tensorflow:loss = 0.00058746594, step = 2201 (0.331 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.166\n",
      "INFO:tensorflow:loss = 0.00021912502, step = 2301 (0.297 sec)\n",
      "INFO:tensorflow:global_step/sec: 255.209\n",
      "INFO:tensorflow:loss = 0.0004376581, step = 2401 (0.390 sec)\n",
      "INFO:tensorflow:global_step/sec: 338.463\n",
      "INFO:tensorflow:loss = 0.00013745876, step = 2501 (0.296 sec)\n",
      "INFO:tensorflow:global_step/sec: 355.2\n",
      "INFO:tensorflow:loss = 0.00015474063, step = 2601 (0.283 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.098\n",
      "INFO:tensorflow:loss = 0.00028257555, step = 2701 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 250.937\n",
      "INFO:tensorflow:loss = 0.00012782379, step = 2801 (0.398 sec)\n",
      "INFO:tensorflow:global_step/sec: 283.216\n",
      "INFO:tensorflow:loss = 1.671609e-05, step = 2901 (0.355 sec)\n",
      "INFO:tensorflow:global_step/sec: 367.068\n",
      "INFO:tensorflow:loss = 3.05287e-05, step = 3001 (0.273 sec)\n",
      "INFO:tensorflow:global_step/sec: 300.584\n",
      "INFO:tensorflow:loss = 6.085734e-05, step = 3101 (0.333 sec)\n",
      "INFO:tensorflow:global_step/sec: 247.562\n",
      "INFO:tensorflow:loss = 0.00028200675, step = 3201 (0.402 sec)\n",
      "INFO:tensorflow:global_step/sec: 330.736\n",
      "INFO:tensorflow:loss = 2.3603425e-05, step = 3301 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 342.4\n",
      "INFO:tensorflow:loss = 0.00016654674, step = 3401 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 333.35\n",
      "INFO:tensorflow:loss = 0.0007090387, step = 3501 (0.302 sec)\n",
      "INFO:tensorflow:global_step/sec: 360.106\n",
      "INFO:tensorflow:loss = 0.00022332428, step = 3601 (0.278 sec)\n",
      "INFO:tensorflow:global_step/sec: 374.7\n",
      "INFO:tensorflow:loss = 5.2743955e-05, step = 3701 (0.267 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.634\n",
      "INFO:tensorflow:loss = 0.000120391494, step = 3801 (0.288 sec)\n",
      "INFO:tensorflow:global_step/sec: 283.204\n",
      "INFO:tensorflow:loss = 0.00016638401, step = 3901 (0.352 sec)\n",
      "INFO:tensorflow:global_step/sec: 343.594\n",
      "INFO:tensorflow:loss = 0.00019014714, step = 4001 (0.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 275.497\n",
      "INFO:tensorflow:loss = 3.9694034e-05, step = 4101 (0.363 sec)\n",
      "INFO:tensorflow:global_step/sec: 312.714\n",
      "INFO:tensorflow:loss = 0.00016885158, step = 4201 (0.319 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.613\n",
      "INFO:tensorflow:loss = 0.00013688122, step = 4301 (0.286 sec)\n",
      "INFO:tensorflow:global_step/sec: 346.379\n",
      "INFO:tensorflow:loss = 3.0980063e-05, step = 4401 (0.289 sec)\n",
      "INFO:tensorflow:global_step/sec: 358.458\n",
      "INFO:tensorflow:loss = 2.172777e-05, step = 4501 (0.280 sec)\n",
      "INFO:tensorflow:global_step/sec: 348.347\n",
      "INFO:tensorflow:loss = 0.00074101484, step = 4601 (0.287 sec)\n",
      "INFO:tensorflow:global_step/sec: 285.804\n",
      "INFO:tensorflow:loss = 0.00057533826, step = 4701 (0.351 sec)\n",
      "INFO:tensorflow:global_step/sec: 316.423\n",
      "INFO:tensorflow:loss = 3.1545118e-05, step = 4801 (0.315 sec)\n",
      "INFO:tensorflow:global_step/sec: 314.556\n",
      "INFO:tensorflow:loss = 8.267825e-05, step = 4901 (0.318 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 5000 into /var/folders/q_/7b87rxq97d74fk63y0b6xcy80000gn/T/tmpa5etbq7o/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 0.0002136573.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.estimator.canned.linear.LinearClassifier at 0x1a1fe28470>"
      ]
     },
     "execution_count": 335,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.train(input_fn=input_func,steps=5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation\n",
    "\n",
    "** Create a prediction input function. Remember to only supprt X_test data and keep shuffle=False. **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 336,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_fn = tf.estimator.inputs.pandas_input_fn(x=X_test,batch_size=len(X_test),shuffle=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 337,
   "metadata": {},
   "outputs": [],
   "source": [
    "#evaluation_input_func = tf.estimator.inputs.pandas_input_fn(x=X_test,y=y_test,batch_size=len(X_test),num_epochs=None,shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Use model.predict() and pass in your input function. This will produce a generator of predictions, which you can then transform into a list, with list() **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 338,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from /var/folders/q_/7b87rxq97d74fk63y0b6xcy80000gn/T/tmpa5etbq7o/model.ckpt-5000\n"
     ]
    }
   ],
   "source": [
    "predictions = list(model.predict(input_fn=pred_fn))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 347,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Starting evaluation at 2018-03-18-13:04:42\n",
      "INFO:tensorflow:Restoring parameters from /var/folders/q_/7b87rxq97d74fk63y0b6xcy80000gn/T/tmpa5etbq7o/model.ckpt-5000\n",
      "INFO:tensorflow:Finished evaluation at 2018-03-18-13:04:43\n",
      "INFO:tensorflow:Saving dict for global step 5000: accuracy = 1.0, accuracy_baseline = 1.0, auc = 0.0, auc_precision_recall = 1.0, average_loss = 1.7344979e-06, global_step = 5000, label/mean = 1.0, loss = 0.01694431, prediction/mean = 0.9999988\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'accuracy': 1.0,\n",
       " 'accuracy_baseline': 1.0,\n",
       " 'auc': 0.0,\n",
       " 'auc_precision_recall': 1.0,\n",
       " 'average_loss': 1.7344979e-06,\n",
       " 'global_step': 5000,\n",
       " 'label/mean': 1.0,\n",
       " 'loss': 0.01694431,\n",
       " 'prediction/mean': 0.9999988}"
      ]
     },
     "execution_count": 347,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eva_fn = tf.estimator.inputs.pandas_input_fn(x=X_test,y=y_test,batch_size=len(X_test),shuffle=False)\n",
    "model.evaluate(eva_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 340,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 340,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Each item in your list will look like this: **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 341,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'class_ids': array([1]),\n",
       " 'classes': array([b'1'], dtype=object),\n",
       " 'logistic': array([1.], dtype=float32),\n",
       " 'logits': array([21.99997], dtype=float32),\n",
       " 'probabilities': array([2.7895533e-10, 1.0000000e+00], dtype=float32)}"
      ]
     },
     "execution_count": 341,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 342,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 342,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions[0]['class_ids'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Create a list of only the class_ids key values from the prediction list of dictionaries, these are the predictions you will use to compare against the real y_test values. **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 343,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = []\n",
    "for pred in predictions:\n",
    "    result.append(pred['class_ids'][0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 344,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 1, 1, 1, 1, 1, 1, 1, 1, 1]"
      ]
     },
     "execution_count": 344,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Import classification_report from sklearn.metrics and then see if you can figure out how to use it to easily get a full report of your model's performance on the test data. **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 345,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 346,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          1       1.00      1.00      1.00      9769\n",
      "\n",
      "avg / total       1.00      1.00      1.00      9769\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test,result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Great Job!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (spinningup)",
   "language": "python",
   "name": "spinningup"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
